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ABSTRACT 


Recent insurgent tactics during Operation Iraqi Freedom (OIF) have demonstrated 
that coalition logistical convoys are vulnerable targets. This thesis examines the tactics, 
techniques and procedures (TTPs) used in convoy operations in an attempt to identify the 
critical factors that lead to mission success. A ground convoy operation scenario is 
created in the agent-based model (ABM) Map Aware Non-uniform Automata (MANA). 
The scenario models a generic logistical convoy consisting of security vehicles, logistical 
vehicles, an unmanned aerial vehicle (UAV) and an enemy ambushing force. The 
convoy travels along a main supply route (MSR) where it is ambushed by a small 
insurgent force. 

We use military experience, judgment and exploratory simulation runs to identify 
11 critical factors within the created scenario. The data farming process and Latin 
Hypercube (LHC) experimental design technique are used to thoroughly examine the 11 
factors. Using the 11 factors 516 design points are created and data farmed over to 
produce 25,800 observations. Additive multiple linear regression is used to fit a model to 
the 25,800 observations. From the created scenario it is concluded that: convoy mission 
success may be determined by only a few factors; the actions of logistical vehicles are 
more critical than those of security vehicles; UAVs provide a statistically significant 
advantage; and ABMs coupled with LHCs and data farming are valuable tools for 


understanding complex problems. 
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THESIS DISCLAIMER 


The reader is cautioned that computer programs developed in this research may 
not have been exercised for all cases of interest. While every effort has been made, 
within the time available, to ensure that the programs are free of computational and logic 
errors, they cannot be considered validated. Any application of these programs without 


additional verification is at the risk of the user. 
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EXECUTIVE SUMMARY 


During Operation Iraqi Freedom (OIF) and the current nation building effort 
taking place, small guerilla type forces ambushed and continue to ambush coalition 
ground convoys. We can continue to expect small attack attempts on logistical convoys 
as long as coalition forces remain in Iraq. “Since the fall of Saddam Hussein, riding 
Iraq's roads has been risky business. U.S. military convoys have been targets for 
ambushes, and armed bandits have cruised the highways” [Riccardi and Sanders, 2004]. 
Furthermore, as the Iraqi’s are learning the lessons of past insurgent conflicts, we can 
assume that future enemies will also learn from the success of Iraqi insurgent tactics. For 


this reason, it is imperative we learn to better defend logistical convoys. 


“Marine Corps training for convoy operations was commonly identified as a 
deficiency, to include training of primary staff members, convoy commanders, drivers 
and participants” [EFCAT, 2003]. Until recently, the protection of a convoy was given 
little thought when planning a logistical movement. Logisticians are now well aware that 
convoy protection lies at the forefront when planning a logistical movement. It is no 


longer a simple optimization problem of supply, demand and vehicle capacities. 


This thesis develops a simulation utilizing an agent-based model (ABM) 
representing a ground convoy operation and identifies the vital parameters that contribute 
to mission success in our scenario. With the use of an agent-based distillation (ABD), 
which reflects the intention to model just the essence of a problem [Anderson, et al., 
2003], this thesis provides insight into the characteristics of the parameters that can be 
varied to increase the probability of convoy mission success. In addition, this thesis 
provides another valuable scenario to support the simulation and data farming processes 
under development by Project Albert (PA). PA is a research and development effort of 
the US Marine Corps Warfighting Lab (MCWL). 


The agent-based modeling environment, Map Aware Non-uniform Automata 
(MANA), was used to model convoy operations. The created scenario depicts a typical 


logistical convoy consisting of security vehicles, logistical vehicles and an unmanned 
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aerial vehicle (UAV). The enemy agents consist of an observer, improvised explosive 
device (IED) and ambushers with individual rifles and rocket-propelled grenades (RPG). 
As the scenario runs, the convoy follows a pre-determined route where the enemy agents 


will eventually ambush them. Figure | displays the baseline scenario in MANA. 
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Figure 1. Baseline MANA Scenario (Best Viewed in Color). 


There are hundreds of factors within a MANA scenario that can be varied to gain 
insights. Based on military experience, judgment and exploratory runs, 11 factors 
believed to be relevant during convoy operations were selected. Each of our 11 factors 
consists of multiple levels, which rules out the possibility of a full factorial experimental 
design. A Latin Hypercube Design (LHC) was used as the experimental design. The 
LHC design is a sampling technique where the goal is to sample the experimental area 
not only on the edges but also in the interior to maximize the space filling of the sampled 
area. [Cioppa, 2002] The LHC technique provides an efficient method to examine many 
factors each with multiple levels in an attempt to uncover any non-linearities within the 
response surface [Brown et. al., 2002]. Supercomputing assets were used to data farm the 
11 factors, each consisting of 33 levels. Each design point is replicated 50 times, 


producing a final data set of 25,800 observations. 
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Linear regression is used to fit a model to the 25,800 observations. The final 
model consists of 14 relevant terms. Through common statistical analysis practices, we 
interpret each of the terms in the final model and relate them to the “real world.” Though 
most of the results found from our research seem to be intuitive, we did find some 


interesting observations that could provide some insights to the tacticians in the field. 


Based on our scenario, the following list summarizes our primary findings: 


° The tactics and actions employed by logistics vehicles within our scenario 
are more of a success determinant than those of the security vehicles. In 
examining the critical term interactions within the model, it was 
discovered that if logistical vehicles remain on the route rather than seek 
concealment, the results are independent of the security vehicle actions. 


e The use of a UAV can provide significant benefits to a convoy by 
providing more situational awareness to the vehicles. 


° Combined situational awareness of surroundings can substantially increase 
the probability of mission success. 


e The more that a convoy operates as a single unit, the greater the chance of 
mission success. 


° The possibility exists that convoy mission success when an ambush is 
encountered can be determined by a few factors. 


e Agent-based models offer us a tool that can help in understanding 
extremely complex problems. 


e The data farming process is an exceptional technique for examining 
numerous variables with expanded levels. 
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I. INTRODUCTION 


A. MOTIVATION 

Over the past year during Operation Iraqi Freedom (OIF) and with the current 
nation building effort taking place, the American people continually hear about the 
ambush of coalition ground convoys by small guerilla type forces. Now that coalition 
forces have transitioned the strategic goal from regime change to nation building and 
occupation, logistical movements have become routine and predictable. We can continue 
to expect small attacks on logistical convoys as long as coalition forces remain in Iraq. 
For the foreseeable future, ground logistical movements utilizing trucks for carrying 
supplies will remain the foundation of the transportation needs in an area of responsibility 
(AOR). While the simple solution to defend against enemy ambushes might seem to be 
heavily armored vehicles with massive amounts of firepower, this solution is by no 
means economically feasible for the time being. Although this thesis topic was motivated 
by recent events in Iraq, the author believes that as the United States (U.S.) Armed forces 
transition to a lighter, faster and more lethal force, the logisticians within the Armed 
forces will be forced to find ways to better defend logistical convoys. It can be assumed 
that future U.S. enemies will learn from the success of the Iraqi ambush techniques. For 
this reason, it is imperative we learn to better defend logistical convoys. 
B. PURPOSE 

The purpose of this thesis is to develop a simulation utilizing an agent-based 
model (ABM) representing a ground convoy operation in an attempt to identify the vital 
parameters contributing to mission success. History states that the enemy will attack 
logistical trains, yet it is still difficult to ensure their security. As seen since the 
beginning of OIF, and continually, as the conflict evolves into a nation building 
operation, convoys are the enemy’s targets of choice. With the use of an agent-based 
distillation (ABD), which reflects the intention to model just the essence of a problem 
[Anderson, et al., 2003], this thesis will seek to provide insight into the characteristics of 
the parameters that can be varied to increase the probability of convoy mission success. 


In addition, this thesis will provide another valuable scenario to support the simulation 


and data farming processes under development by Project Albert (PA). PA is a research 
and development effort of the U.S. Marine Corps Warfighting Lab (MCWL) and will be 
discussed further in a later section. 
C. BACKGROUND 

In the current global atmosphere, no nation can match the combat power of the 
U.S. Armed forces in a head-to-head engagement. This past year has witnessed the Iraqis 
continuously attacking ground logistical convoys, indicating the adoption of a tactic of 
harassment in an attempt to prolong operations. Known as a “Fabian Strategy,” a weaker 
force will avoid decisive battle with a more powerful or skillful enemy. While avoiding 
decisive battle, the side employing this strategy harasses its enemy to cause attrition and 
loss of morale. Employment of this strategy implies that the weaker side believes time is 
on its side, but it may also be adopted when no feasible alternative strategy can be 
devised [Hart, 1967]. This strategy derives its name from Quintus Fabius Maximus, who 
defended Rome against Hannibal in the Second Punic War (218-201 B.C.). The colonists 
during the American Revolution, the Confederacy during the American Civil War, the 
North Vietnamese Army (NVA) during the Vietnam War, utilized the strategy. 
Currently, it is evident that the Iraqis have adopted this strategy as a last resort to weaken 
the resolve of the American people and force the U.S. Government to withdraw from 


Iraq. 


With future United States Marine Corps (USMC) doctrinal concepts such as “Sea- 
Basing,” Operational Maneuver from the Sea (OMFTS) and Ship to Objective Maneuver 
(STOM) lurking on the horizon of the next decade, battlefield distribution planning 
becomes more crucial than ever before. Whether on the ground or at sea, the convoy 
operation is, and will continue to be, the primary means of transportation and a critical 
part of the overall logistical plan. Tactics used during OIF provide some insight into 
what the future holds for the U.S. war fighting playbook and identifies some of the 
problems we are likely to encounter. One such example, as the combat element moves 
quickly to reach ultimate objectives, we can probably expect the tactic of bypassing low 
threat enemy positions to continue to be utilized. This will no doubt cause great 


headaches for the logistical planners as they struggle to produce solutions for supplying 


the fast moving combat elements while simultaneously protecting logistical trains from 
those bypassed threats. With the use of an ABD, this thesis seeks to gain insight into 


how we can best defend logistical trains. 


Recognizing that the current enemy, and most likely any foreseeable future 
enemy, will shy from decisive engagements, it is possible to use history as a guide to 
speculate that they will seek out the “soft targets.” There are numerous “soft targets,” but 
the obvious target is any type of logistical convoy supporting the forward combat troops. 
The simple nature and composition of a convoy makes it a vulnerable target. A convoy is 
typically slow and cumbersome with logistical vehicles that are lightly armored and 
armed, organic security and firepower tends to be disbursed, fire from any supporting 
arms may be limited, and it takes the enemy minimal resources to cause damage to the 
convoy. Support of large combat maneuvers requires enormous amounts of fuel, 
ammunition, rations, and so forth. Consequently, numerous convoys on the road offer 
the enemy multiple targets. After action reports, lessons learned reports and the history 
books from OIF, Vietnam and other past wars consistently reference the Tactics, 
Techniques and Procedures (TTPs) used during convoy operations as problem areas. 
During the United States Civil War both the Union and Confederate forces would destroy 
rail lines cutting the Lines of Communications (LOC) to the combat forces. During 
World War II (WWII), the Germans used U-boats to attack the weakly defended supply 
ships in the Atlantic. More recently and relevant to this thesis, the North Vietnamese 
Army (NVA) used ambushes to attack logistical convoys with great success. The 
Vietnam reference is particularly relevant. The NVA tactics were similar to the Iraqi 
tactics currently being employed. For example, the U.S. Army’s 3" Transportation 
Group was frequently ambushed up until the winter of 1968 when they were forced to 
harden logistical convoys with the use of the somewhat homemade “Gun Trucks” to 
increase the amount of security within the convoys. This is interesting, as the U.S. Army 
is currently tinkering with this idea almost forty years later. The fact that there are 
numerous historical references regarding convoy security coupled with the reality of the 
continual struggle to defend the convoys provides the purpose for applying modern 


modeling techniques to this problem to gain insight into convoy protection. In order to 


minimize both human and equipment losses, it is necessary to harden the targets in the 
area of operations and deny the enemy the ability to project its will on the United States’ 
strategic goals. 
D. SCOPE 

During the course of a ground convoy operation, it is possible to examine 
hundreds of parameters that might contribute to the success of the operation. Obvious 
items of interest may include equipment, tactics, communications, speeds and enemy 
actions. In an effort to confine this thesis to issues of reasonable examination, we refer 


the reader to the following analysis plan: 


° Identify possible enemy courses of action, including known weapons 
utilized. 

e Review practiced Tactics, Techniques and Procedures for convoy 
operations. 

e Identify the appropriate measures of effectiveness. 

e Explain the capabilities of the chosen modeling environment. 

° Examine the construction of our ground convoy simulation model. 

e Present the design of experiment for the simulation model. 

e Explain data farming and its application to our ground convoy simulation 
model. 

e Analyze and fit models to data output from the simulation runs. 

° Examine the results of the data analysis and its usefulness in conducting 


ground convoy operations. 

E. AGENT-BASED MODELS 

Combat operations take place in an extremely dynamic environment where the 
human element plays a more prominent role than the technologies utilized. While 
technology and the weapon systems utilized are the means to an end, human interactions 
control these technologies and determine the ultimate outcome of a conflict. Unlike the 
traditional combat models that utilize physics and historical data to determine outcomes, 
ABMs attempt to model the human decision making element. Relatively new in the 
world of combat simulation, ABMs provide a tool that attempts to capture complex 


interactions between individual entities on the battlefield. 


ABMs center on the individual entity, known as the agent and are not 
programmed to react in a prescribed manner. Rather, they are setup to have tendencies 
towards a certain reaction or behavior. The power of ABMs lies in the ability to quickly 
create scenarios, examine the outcomes of the adaptive properties of the agents and 
compile numerous scenario runs quickly. For this reason, ABMs appear to be ideally 
suited for modeling convoy operations in an unknown hostile environment. 

F. PROJECT ALBERT 

PA is a research and development effort of the United States Marine Corps 
Warfighting Lab, a branch of the Marine Corps Combat Development Command 
(MCCDC) in Quantico, Virginia. PA utilizes a vast array of experimental modeling tools 
to explore questions posed by military decision-makers. The goal of PA is to apply data 
farming techniques to an ABD in an attempt to explore the intangible factors of combat 
that influence a commander’s decision-making process [Horne and Johnson, 2002]. Data 
farming is a method of investigating a large number of factors over a wide range of 
values. This technique allows the modeler to explore thousands of combinations of 
factors in an attempt to relate simulation outcomes to real life problems. For the record, 
PA does not proclaim to possess the ability to develop and run models for the purpose of 
prediction or to identify final answers to problems. Rather, PA seeks to gain insight into 
a problem through exploring a multitude of possible outcomes. For information on PA 
and past research done by PA, refer to their website at www.mcwl.quantico.usme.mil. 

G. GROUND CONVOY SIMULATION 

The ground convoy simulation model created for this thesis will be fully 
described in a later chapter. To give the reader an idea of the composition of the 
simulation, a brief description is provided to ease in understanding the following problem 
description. The creation of the ground convoy simulation used Map Aware Non- 
uniform Automata (MANA), an ABM, as the modeling environment. The screenshot in 
Figure 2 displays the simulation in the midst of a simulation run. Blue trucks and 
armored personnel carrier (APC) icons represent the friendly convoy. Blue trucks are 
logistical vehicles while the APC icon represents the security element of a convoy. The 


red cross on the road represents an improvised explosive device (IED), the red artillery 


piece is an enemy rocket propelled grenade (RPG) type weapon and the red infantry icons 
are enemy agents with AK-47’s. Detailed descriptions of the agents and information on 


how the scenario executes appear in a later chapter. 
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Figure 2. Ground Convoy Model (Best Viewed in Color). 


H. THE PROBLEM 

“Marine Corps training for convoy operations was commonly identified as a 
deficiency, to include training of primary staff members, convoy commanders, drivers 
and participants.” [EFCAT, 2003] Until recently, the protection of a convoy was given 
little thought when planning a logistical movement. Over the past year, logisticians have 
been made aware that the protection of the convoy now lies at the forefront of movement 
planning and is no longer a simple optimization problem of supply, demand and vehicle 
capacities. While it might seem awkward for logisticians to be concerned with the tactics 
of combating an enemy ambush, it is pertinent because the logisticians are planning and 


conducting the majority of the convoy operations. During the course of a convoy 
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operation, a convoy commander faces numerous decisions that can enhance the chance of 
success or lead to failure. While there are numerous decisions made during the operation, 
this thesis will focus only on those decisions made upon encountering an enemy ambush. 
Both the Army and the Marine Corps teach tactics for combating an enemy ambush at the 
Motor Transport schools. New technologies and an elusive enemy with less than 
traditional goals during the ambush engenders whether other ways might exist to employ 
these new technologies coupled with different tactics to defend these convoys better. 
This thesis will attempt to capture these loosely defined parameters in an ABD to try to 
gain insight into the critical parameters surrounding a convoy operation in a hostile 
environment. This thesis will apply data farming techniques to an ABD that models 
ground convoy operations during an ambush in an attempt to determine which 
parameters, such as security vehicle locations within a convoy, aggressiveness of security 
and the desire of a convoy to drive through an ambush or to stop, have the greatest effect 
on mission accomplishment in our scenario. 
1. THESIS ORGANIZATION 

Chapter II begins by detailing the basics of Marine Corps convoy doctrine, some 
of the currently practiced methods, and finally, some unclassified enemy tactical 
observations. Chapter III covers a detailed description of MANA and an in-depth 
description of how it works. Next, a complete description of the convoy simulation 
created for this thesis is discussed and how the parameters chosen relate to the “real 
world.” This chapter also discusses the results from the base case convoy model runs. 
Chapter IV discusses the analysis methods used in interpreting the results of the model. 
Chapter V explains the simulation results and the models fitted. Chapter VI completes 
the thesis with a discussion of conclusions from the simulation and final model 


interpretations. This chapter also presents recommendations for future research. 
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I. DOCTRINAL PRINCIPLES OF CONVOY OPERATIONS 


“Determine what improvements to training and doctrine are needed to provide 
adequate convoy operations training and TTPs, to include security measures” [EFCAT, 
2003]. The quote was listed as an area for further study in the reference. This chapter 
will begin by summarizing some of the historical insurgent tactics, identify relevant 
doctrinal convoy procedures, then move on to some of the observed enemy tactics being 
employed in the AOR. It should be noted that in an attempt to keep this thesis 
unclassified, current enemy tactics are covered vaguely from unclassified sources. For a 
more detailed exploration of some of the enemies TTPs, the reader is directed to relevant 
SIPRnet websites. In the interest of brevity, we will only cover doctrinal convoy 
practices that are pertinent to this thesis. For a complete listing of all doctrinal 
procedures, the reader is directed to the US Marine Corps Publication MCRP 4-11.3F, 
Convoy Operations Handbook. 

A. INSURGENT TACTICS, TECHNIQUES AND PROCEDURES (TTP) 

Though most of the reports detailing enemy ambushes in Iraq remain classified, 
we will attempt to identify some of the observed enemy tactics from unclassified sources. 
It has become evident that the enemy insurgents seem to be far more organized than 
previously suspected. It also appears that the insurgents may be applying some of the 
ambush techniques used by the Chechnyan’s against the Russians and by Al-Qaida and 
the Taliban in Afghanistan against U.S. Forces, observed by a US Army 3™ Corps 
Intelligence Officer [Krane, 2004]. If the Iraqi insurgents are utilizing historical guerilla 
tactics, then it is also safe to assume they are following the tactics of other past guerilla 
type forces. In this section, we will first cover some observed historical insurgent or 
guerilla tactics and some of the current observations from Iraq relating to convoy 
ambushes. 

1. Historical Guerilla Tactical Observations 

The Iraqi tactics currently being employed are characteristic of observed guerilla 
tactics of the recent past. Guerilla warfare can be defined as a set of tactics used by a 
minority group within a state to oppose a government or occupying force [Ipecki, 2002]. 


In most cases, it seems the goal of a guerilla force is to prolong a conflict, weakening the 
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resolve of their opponent. The guerilla force uses terrorism and harassment to 
accomplish their goals. With extensive knowledge of their environment and terrain, they 
tend to be far more mobile than their enemy. They tend to be extremely self-reliant with 
little dependence on formidable lines of communication (LOC). Even in the face of 
ultimate defeat, they possess a strong will to carry-on the fight at all costs. In the 
provided reference, six 20" century guerilla conflicts were examined; some of the 


pertinent observations are listed below [Ipecki, 2002]: 


e Guerillas tend to move and fight at night. 

e They fight mostly offensively avoiding defensive strongholds. 

e Guerilla attacks are typically conducted at close range. 

e They rely on shock, surprise, speed and intelligence. 

e They tend to attack from unpredictable positions, utilizing their vast 
knowledge of the terrain and environment to acquire speed and surprise. 

e Guerillas typically gain intelligence through unconventional means having 
extensive communications and observation networks within the local 
populace. 


The reference was written prior to OIF and did not examine the Iraqi tactics. 
However, one can look at the list and relate most of them to the current situation in Iraq. 

De Observed Iraqi Tactics Against Convoys 

It is estimated that 1-5 percent of coalition convoys are attacked daily [181* 
Trans. Bn, 2003]. It is apparent that we are not dealing with a static insurgent group; the 
enemy is constantly evolving to counter our tactics and our use of modern technology. 
The insurgent’s attacks are complex, well coordinated and are increasing in both number 
and sophistication. A presentation from the 181“ Transportation Bn, lists some of the 


observed Iraqi tactics: 


* The enemy insurgents are not easily identifiable; they seek to blend in 
with populace. 

e They are often attacking the “soft” targets. 

® They are using IEDs, small arms and RPGs in the attacks. 

° They avoid decisive and sustained contacts. 

e They attack the rear of convoys 70 percent of the time. 

° They are practicing “hit-and-run” tactics. 
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e They are utilizing an extensive scout network to monitor convoy 


movements. 

e The insurgent TTPs are constantly evolving; they select convoys which 
provide maximum damage to coalition. 

® Their methods for IED employment is constantly changing. 

° They seek to disrupt convoys with vehicles, civilians and natural hindering 
points of routes in attempt to slow the convoys down. 

° The ambushes are typically command executed from observers. 

@ The small arms and RPG fire is usually directed at lead or trail vehicle. 


The reader is advised that this is not an all encompassing list, specifically because 
the enemy continues to evolve. However, the insurgent’s goals when an ambush is 
conducted seem to be remaining constant. They seek to stop movement, isolate desirable 
targets and kill coalition forces in an attempt to demoralize and weaken our resolve. 

3. Coalition Convoy Defensive Tactics 

It is assumed that current coalition tactics are probably somewhat different based 
on the unit, environment, terrain and organic assets. Since it is impossible to ascertain 
how every unit in the AOR is operating, we will briefly summarize some of the TTPs 
being practiced from the limited unclassified sources. 

a. Convoy Layout 

The 181* Transportation Battalion dictates having an experienced driver 
in the lead vehicle with some sort of gun truck or security vehicle directly behind the 
lead. Gun trucks were used extensively during the Vietnam conflict and have resurfaced 
in recent months to combat the growing threat to convoys. A gun truck in most cases is 
simply a logistical vehicle such as a 5-ton truck with automatic weapons mounted in the 
bed. Not only do they possess increased firepower but also act as a deterrent based on 


their intimidating posture. Figure 3 is a picture of one type of gun truck. 
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Figure 3. Vietnam Era Gun Truck (From: Army Transportation Association). 


The convoy commander’s vehicle should be placed centrally within the 
convoy to allow for enhanced command and control (C’). The second security vehicle 
should be placed directly behind the commander’s vehicle to guard flanks and act as a 
reserve if necessary. A third security vehicle should be the last vehicle in the convoy to 
enable over-watch of the entire column. All other combat logistic vehicles should be 
spread evenly throughout the serial. A serial is the name given for a single convoy. 
There should be 2-3 security vehicles per convoy and convoys with more than 10 
vehicles should have 3 security vehicles. A few other current practices are listed below: 

e No headlights on, only running lights. 


° Take quickest lane or lane that allows for easiest travel. Reference claims 
that IEDs are statistically ineffective on vehicles moving faster than 40 
mph with 50 meter dispersions between vehicles. 


° Attempt to stay in middle of roads away from sides where IEDs could be 
placed. 
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e Gun trucks display tough and offensive posture. 
° Gun trucks aggressively suppress small arms fire. 
e Every member of a convoy is responsible for security. 

Again, these tactics are strictly limited to the unit for which information 
was available. Research of other similar presentations by both the US Army and Marine 
Corps units report similar tactics and all seem to be somewhat typical and do not alter 
much from the provided doctrinal practices. During a visit to the I Marine Expeditionary 
Force (MEF) it was also stated that convoy sizes do not tend to exceed 20 vehicles in the 
current environment [Widdowson, 2004]. During the offensive phase of OIF, convoys 
were in excess of 20 vehicles as the combat units made their way from Kuwait to 
Baghdad. This thesis is examining the current operating environment. 

B. MARINE CORPS DOCTRINE 

In this section, we will cover prescribed movement formations, placement of 
vehicles, and actions upon altering types of ambushes. 

1. Vehicle Movement 

Logistical convoy movements are typically organized in some type of column 
with altering amounts of dispersion. Dispersion of vehicles is noted as of critical 
importance when trying to minimize the effects of an enemy attack. Some important 
considerations are that it is difficult for an enemy to bring large amounts of fire to bear on 
vehicles that are widely dispersed, mine damage is localized when dispersed and the 
efficiency of an enemy air attack is minimized. Convoy speeds are prescribed to be 5-10 
miles per hour (mph) below the posted road speed limits depending on road conditions, 
weather conditions, vehicle capabilities and urgency of move. Listed below are the three 
doctrinal formations [MCRP 4-11.3F, 2001]. 


e Open Column — Used during daylight hours, vehicle dispersion is typically 
50-100 meters, with a vehicle density of 10-15 vehicles per kilometer. 


e Close Column — Used in darkness and when visibility is limited. 
Dispersion is typically 25 meters. 


e Infiltration — Movement of dispersed, individual vehicles or units at 
irregular intervals. Prevents undue massing of vehicles and reduces traffic 
density. Used when security, deception and dispersion are the goal. This 
technique provides a passive defense against enemy observation. 


13 


Immediate Action Drills 
a. Snipers 


According to the Convoy Operations Handbook, snipers alone can do little 


damage to moving vehicles. However, a sniper can be used to try and stop a convoy and 


a larger 
follows: 
e 
e 
e 


ambush could follow. Actions to take when a sniper is encountered are as 


Do not stop. 
Throw smoke to screen enemy observation. 
Provide suppressive fire. 


Be vigilant of potential future confrontations. 
b. Air Attacks 


Though enemy air attacks are not presently a concern the following is 


provided. Enemy aircraft will attempt to fire along the long axis of a convoy in an 


attempt to cause maximum damage on a single pass. Vehicles should drive off the road 


in a herring bone formation seeking concealment and attempt to return fire. The herring 


bone formation is displayed in Figure 4. 
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Figure 4. Herring Bone Formation (From: MCRP 4-11.3F, 2001). 
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c. Ambushes 
Accordingly, the enemy ambush is described as the single greatest threat 
to a convoy’s mission success and survival [MCRP 4-11.3F, 2001]. Enemy ambushes are 


categorized in two areas: unblocked and blocked. 


During an unblocked ambush, vehicles caught in the kill zone should 
continue to move through. Vehicles that have yet to reach the kill zone should stop, seek 
cover and concealment and dismount. Armored security vehicles should find positions 
that enable providing suppressive fire and support the maneuver of the security forces. 
Security forces should then close with and assault the enemy based on rehearsals. Close 


air support and indirect fire assets should also be called in on the enemy. 


In the case of a blocked ambush, vehicles not yet in the kill zone should 
stop, seek cover and concealment and dismount. Vehicles that are caught in the kill zone 
should conduct unloading drills and return fire and/or assault the enemy. Security 
vehicles seek positions to return fire and provide support for security forces maneuvering. 
Security forces should maneuver from outside the kill zone and assault enemy positions. 

3: Convoy Security Forces 

A convoy security element will typically consist of some type of armored vehicle 
with a mounted machinegun or MK-19 grenade launcher. Vehicles could be Amphibious 
Assault Vehicles (AAV), tanks or, more likely, armored High Mobility Multipurpose 
Wheeled Vehicles (HMMWV). MCRP 4-11.3F states that the mechanized vehicles, such 
as AAVs, are preferable as escort vehicles due to their firepower and protection from 
mines, indirect and direct fire. Though some type of mechanized vehicle would be ideal 
as an escort, the reality is that rarely will a Combat Service Support (CSS) unit have 
access to mechanized vehicles. It is conceded that every situation is different and there 
are probably incidences when this is not true, however, this thesis assumes that escort 
vehicles are armored HMMWVs. 

a. Security Element Tactical Disposition 

The number of escort vehicles per convoy in reality is probably based 
more on available assets than on doctrinal publications; however, MCRP 4-11.3F states 
that each escort vehicle should protect 5 to 10 convoy vehicles. Security during a convoy 


must be throughout the entire convoy and in all directions. For this reason security 
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should be evenly dispersed amongst the convoy vehicles. A fire support team (FIST) 
should be located near the convoy commander and engineer assets near the front of a 
formation in order to deal with obstacles. A column formation as depicted in Figure 5 


will normally be utilized due to its inherent speed and ease of movement. 


Engineer 
Squad 


Pipe PR = BBR 


Figure 5. Column Formation. Teams A, B and C are an example of the security 
element dispersed throughout the column. Convoy blocks represent multiple logistics 
vehicles (From: MCRP 4-11.3F). 

b. Security Element Actions Upon an Ambush 

As mentioned before, the enemy ambush is the single greatest threat to the 
survival of a convoy, therefore the security element must be prepared to respond quickly, 
overwhelmingly and decisively. Security vehicles must be well rehearsed and clear on 
their responsibilities if an ambush is encountered. The following are escort vehicle 


actions when an ambush is encountered [MCRP 4-11.3F, 2001]: 


e Upon enemy detection, escort vehicles seek covered positions between the 
convoy and enemy providing the maximum amount of fire possible (see 
Figure 6). 

e The remainder of the convoy continues to move along the route at the 
highest possible rate of speed. 

e Armed convoy vehicles return fire until escort or security vehicle is in 
position. 


° Damaged or disabled vehicles should be abandoned and pushed off the 


route. 
e If available, escort vehicle leader will call for fire support. 
e Once the convoy is clear of kill zone, escort vehicles will either, based on 


escort composition and enemy strength, continue suppressive fire while 
waiting on additional forces, assault the enemy or break contact and move 
out of kill zone. 
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The Convoy Operations Handbook goes on to explain actions the security 
element takes during halts, obstacles and how to provide area security. These details will 


not be covered due the fact they are not relevant to the scenario created for this thesis. 


We have briefly discussed some of the historical aspects of insurgent 
groups, some of their observed tactics, doctrinal principles of convoy operations and 
finally some of the observed Iraqi tactics as well as some coalition TTPs being practiced 
to combat the evolving threat. It is clear that our current enemy is extremely intelligent 
and adapts to the environment. It is also evident that they are learning from past 
conflicts, it is imperative that our forces take advantage of the mass amounts of lessons 


learned files compiled from previous conflicts. 






Scouts action toward ambush 
and suppresses enemy. 


B. Convoy continues to move 
at increased speed, 


Figure 6. Security Actions Upon Ambush (From: MCRP 4-11.3F) 
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Hil. MODEL DESCRIPTION 


The agent-based modeling environment, MANA, was chosen for this thesis to 
model convoy operations. This chapter covers MANA in detail including; the history, 
the purpose, why it was chosen as the modeling environment and key features within 
MANA. In the interest of brevity, we will cover the key features of MANA are covered 
as they relate to our ground convoy scenario. 

A. MANA 

One of the many desirable aspects of MANA is the easily understandable user’s 
manual that accompanies each new version of the program. Most of the information 
relating to MANA in this chapter is taken from the April 2004 version 3.0.35 user’s 
manual. 

1. MANA Overview 

“Tt’s the unconquerable soul of man, and not the nature of the weapon he uses, 
that ensures victory.” [General George S. Patton] This quote simply summarizes why 
the MANA developers were frustrated with conventional physics-based combat models 
and began exploring ways to model the intangible aspects of combat. Many aspects of 
combat can be modeled using physics, such as; the impact point of a projectile, the 
explosion radius of an artillery round or the amount of fuel needed to get an aircraft into 
flight. Physics-based models cannot model the behavior of troops in combat which can 
be argued to be far more decisive in combat than the knowledge of where a round 
impacts. In the conventional models such as JANUS, the entities’ decisions are 
programmed and completely deterministic in nature. These facts led the developers to 
the following observations, 

Behaviour of troops in any kind of scenario plays an important role, but it 

is often overlooked by analysts because human nature is a mathematical 

intangible, just as is the weather. As with many other intangibles in 

analytical combat models, it is often thrown away in favour of dwelling on 
infinitesimal details, often of little real importance to a given scenario. 

Generally speaking, there seems to be a school of analysts who believe 

that just because they have an equation to describe some aspect of a 

scenario, then that aspect must be more “real” than the aspects of the 


scenario that cannot be so easily described. We would argue that is 
precisely the aspects that are most difficult to describe which are of the 
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greatest importance. So much in war is intangible, and these intangibles 
are spoken about so often it is impossible to ignore them [Anderson et al., 
2004]. 


MANA is based on two key ideas [Anderson et al., 2004]: 


e That the behaviour of the entities within a combat model (both friend and 
foe) is a critical component of the analysis of the possible outcomes. 


e That we are wasting our time with highly detailed models for determining 
force mixes and combat effectiveness [Anderson et al. 2004]. 


MANA was developed by New Zealand’s Defence Technology Agency (DTA) 
with initial work beginning in February 1999 after the US Marine Corps chief scientist 
Dr. Alfred Brandstein and colleague Dr. Gary Horne made a visit to DTA. During the 
visit, DTA was introduced to the models ISAAC and EINSTein. Using these models 
DTA explored how automation models could be used for analysis. In 2000, DTA used a 
program in MATLAB in an attempt to increase the sophistication of the automation 
models using a concept of a “situational awareness map.” Later in 2000 it was 
discovered that the MATLAB environment lacked the programming flexibility to meet 
the task at hand. DTA then turned to building a new model in the Delphi Object Pascal 
Language. The result is the MANA model. Many of the original concepts in MANA 
were derived from the ISAAC model [Anderson et al., 2004]. 

2; Why MANA 

MANA was developed with the ability to quickly set-up scenarios and observe 
global behaviors that emerge from many local non-linear interactions between entities. 
“MANA is not intended to be able to describe every aspect of a military operation. 
Furthermore, there is no built-in “intelligence” that determines the plan that MANA 
entities are working towards. Consequently, careful thought must be given when setting 
up a scenario” [Anderson et al., 2004]. The MANA user must have a particular aspect of 
combat in which the designed scenario is modeling and what the agents are trying to 
accomplish. Although this might seem to add an element of scripting “the non-linear 
nature of the model ensures that, regardless of the modeller’s preconception, a startlingly 


large number of outcomes are possible” [Anderson et al., 2004]. 


MANA is not the answer for all combat modeling and does have some limitations. 


One such limitation is in the exploration of strict combat formations. Formations are 
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difficult to model in MANA, and will typically appear in a scenario by accident rather 
than by design. Though this might seem to be a problem at first, these occurrences add to 
the randomness of human behavior and can be expected in “real life” [Anderson et al, 
2004]. MANA offers the ability to quickly examine many different ways of doing things 
rather than one perfect or supposed perfect way. For this reason MANA lends itself well 
to modeling convoy operations and examining the outcomes of many different 
combinations of factors. Logistical convoy formations are few in practice, typically a 
single file line of vehicles, but the actions of the vehicles when enemy contact is 
encountered are infinite. MANA enables us to vary numerous factors controlling the 
convoy in an attempt to identify what combinations of factors seem to produce the best 


results in the MANA scenario. 


Ultimately, MANA was chosen for this thesis for several reasons. First, there are 
an infinite number of factors which contribute to the success or failure of a convoy, 
MANA provides an environment in which many of these factors can be varied. Second, 
the actions that a convoy might take when enemy contact is encountered, is described in 
the simulation community as a “change in state.” MANA provides an extensive ability to 
alter agent desires when a state change is encountered. Third, MANA is extremely user 
friendly and easily learned from the user manual and by trial and error. Finally, the 
purpose for why MANA was created grasps the essence of what this thesis hopes to 
identify. There are many physical aspects of a convoy that could probably be modeled 
with physics-based simulation environments, such as, armor and weaponry, but a 
conventional model would have difficulty combining these two factors with numerous 
others to identify global effects of many local relationships. 

B. FEATURES OF MANA RELATED TO OUR SCENARIO 

Here we will go into detail describing the inner workings of MANA and how they 
are used to construct our ground convoy model. The MANA user has the capability to 
alter numerous items in a scenario; we will only cover those items which are used for our 
scenario. In this section, we will cover squad creation and those created for our scenario. 
Parameters that are varied in our scenario, including their ranges and relation to the “real 
world,” are listed in Appendix A. For a complete listing of MANA features, the reader is 
referred to the MANA version 3.0.35 user’s manual. 
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1. MANA Settings Tabs 

When building a scenario in MANA there are nine tabs that the user must be 
familiar with in order to create a scenario. These will be detailed below. Within the tabs 
are four main areas where settings can be made to determine how a squad acts, shoots, 
communicates and moves. The reader is referred to Figure 10 where at the top of the 
figure the nine settings tabs are displayed. 

2. Situational Awareness (SA) 

Before we attempt to explain our scenario and the inner workings of MANA it is 
important to explain the squad SA and inorganic SA concepts. The SA map displayed in 
Figure 7 is used to simulate communications between agents and squads. Squads are 
comprised of individual agents with identical properties; multiple squads can be created 
that are friendly to each other using the allegiance settings. Memory is shared between 
squad agents using the situational map. The map is updated instantaneously when a 
squad member detects other agents. The situational awareness map acts as a collective 
picture of sensor information within the squad. Information on the squad’s SA map can 
be shared with other squads using the inter-squad communications options. Information 
passed via inter-squad communications are stored on a separate map, known as the 
inorganic situational awareness map displayed in Figure 8. Squad agent’s personality 
parameters can be created to determine responses based on information from both the 
squad SA map and the inorganic SA map. MANA utilizes a color and geometric figure 
scheme to portray detections on the map. Squad friends are shown as inverted triangles, 
other friends as upright triangles, enemies and unknowns as colored and white boxes 
respectively and finally neutrals by diamonds. Colors red, yellow and light greys are 
used to identify a contact’s threat level, which is specified by the user. The inorganic SA 
map shown in Figure 8 is similar to the SA map, the difference being that there is no 
“Show Squad Friends” option. This is removed due the fact that an agent or squad 
receiving third party contact information cannot confirm that a contact is a squad member 


[Anderson et al., 2004]. 
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Figure 7. Squad Situational Awareness Map (Best Viewed in Color). 
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Figure 8. Squad Inorganic Situational Awareness Map (Best Viewed in Color). 
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35 Workings of Our Ground Convoy Scenario 

Figure 9 displays the base scenario created for this thesis. The terrain represents 
the outskirts of an urban environment. There are no elevation changes in the terrain other 
than man-made structures represented by the grey polygon figures. The buildings cause 
line of sight and movement problems for the blue convoy. In an attempt to simulate the 
enemy having the advantage of home terrain, the buildings do not affect their movement 


or line of sight. 


The blue convoy starts at the bottom left of Figure 9 and follows a prescribed 
route to the top right side of the screen. The scenario runs for 4000 time steps. This is 
not altered in any of the runs. The blue convoy consists of seven MANA squads, two 
squads of logistical vehicles, three squads of security vehicles, one unmanned aerial 
vehicle (UAV) and one squad designated as an agent to kill the red observer. The red 
forces consist of four MANA squads, one squad of infantry, one squad with a rocket 
propelled grenade (RPG), one squad depicting an observer or scout and one squad that 
acts as a stopping condition for the scenario. Each run begins with the blue vehicles in a 
relatively similar location; red forces are placed somewhat randomly within a specified 
area of the screen. Each squad’s parameter settings will be examined more thoroughly 


later in this chapter. 


At the start of a simulation run, the blue convoy moves across the bottom of the 
screen into the more urban area following the yellow road to the top right of the map 
depicted in Figure 9. The red agent depicted by a telephone pole icon represents an 
enemy observer. The observer’s purpose is to detect a blue convoy moving along the 
route, alert other ambushers and arm an JED placed on the road, represented by an 
addition sign. Once the ambushers are alerted of the convoy they will move towards the 


road and attempt to fire on the blue agents. 
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Figure 9. Ground Convoy Scenario (Best Viewed in Color). 


4. MANA Squads 

A squad in MANA is a group of agents with size set by the user from 1 to 1000. 
Agents within a squad share the same properties and can be set up to change properties in 
different states, where a state is simply a set of parameter values that determine an 
agent’s desires or behaviors. The parameters altered for our squads are covered in 


Appendix A. The squads created for this thesis are listed in Table 1. 
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Squad Squad Name Allegiance # of Description Icon 
# Agents 
1 Red Infantry Red 5 Infantry Soldier 
with AK-47 
2 Blue Back Blue 6 Logistics Truck 
vehicles 
2 IED Red 1 Explosive Addition 
Device sign 
4 BlueUAV Blue 1 UAV Airplane 
5 RedRPG Red 2 Infantry Artillery 
with RPG piece 
6 BlueFront Blue 6 Logistics Truck 
vehicles 
fi Stop Condition Red 1 Indicates Soldier 
when Blue 
reaches goal 
8 BlueSecurityFront Blue 1 Armored APC 
HMMWV 
9 BlueSecurityBack Blue 1 Armored APC 
HMMWV 
10 BlueSecurityMiddle Blue 1 Armored APC 
HMMWV 
11 RedObserver Red 1 Scout Telephone 
Pole 
12 ObserverKiller Blue 1 Fires on Red Blue 
Observer artillery 
piece 
Table 1. Ground Convoy Squads. 


Figure 10 displays the general properties tab of MANA, where the creation of a 


squad begins. On this screen, squads are given names, the number of agents in the squad 


is established and set to active or inactive. Also on this screen the user may perform 


other administrative type tasks such as saving, copying and loading squads. 
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Figure 10. General Properties Tab. 





5. MANA Terrain 

MANA terrain is based on pixels or cells which can vary in number from 50° to 
1000? pixels; the default terrain is a 200 x 200 grid, our scenario uses a 1000 x 1000 pixel 
grid. A MANA map file is based on a standard Windows bitmap. Cell colors can be 
altered to influence agent movements. The default battlefield provides five preset 
terrains that can be altered to affect movement. Cells can be manipulated to restrict or 
allow free movement, provide cover and concealment, affect lines of sight and act as 
barriers. In previous versions of MANA each cell could only be occupied by a single 
agent, this is no longer the case; the user has the option of allowing more than one agent 


in a cell. 


Though MANA provides five preset terrains, the user can specify other terrains to 
achieve a desired movement or line of sight effect. We will cover the five preset terrains 


to give the reader an idea of how terrains affect the agents in a given scenario. 
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e Billiard Table — Default terrain that has no special properties affecting 
agents, movement and line of sight are unrestricted. Appears black in 
color. 


° Easy Going — Terrain representing roads or other regions that are 
particularly desirable for agents to move along. Easy going terrain 
appears yellow in color. Agents can be created to have a desire to move 
along an easy going terrain. 


e Wall — Acts as an obstacle for agents and appears grey in color. Agents 
can not occupy a cell that is set as an obstacle. Obstacle cells can also 
restrict line of sight if this option is turned on. 


e Light Brush/Dense Brush — Acts as vegetation and appears green in color. 
Altering the density can affect speed of movement as well as providing 
cover and concealment from fire and observation. 


e Hilltop — Terrain that offers high degree of concealment appearing dark 
grey in color. 


It is important to reiterate that the user has the ability to specify their own terrains 
establishing how the terrain affects the agent. Agents can also be parameterized to either 
desire or avoid specific terrains. Figure 11 displays the MANA map tab where all terrain 


modifications are made. 
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Figure 11. Map Tab (Best Viewed in Color). 


The scenario map is displayed here giving the user the capability to establish 
starting points for each squad. The user may manually enter (X, Y) coordinates for 
starting locations or simply point and click with the mouse. The user also has the ability 
to establish exact points for the start locations or to have the agents placed randomly 
within an area. Once starting locations are established for each squad, waypoints can 


then be established for the agents to move towards if desired. 


In our scenario, the convoy agents are placed on the left hand side of screen along 
the road in a linear formation. Each blue agent is placed in nearly the same location at 
the start of each run. Waypoints were established along the road identifying the convoy 
route. Although this seems to predetermine where the blue agents will go, this is reality. 


Before any real convoy hits the road, a preplanned route and order of march will be 
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established [MCRP 4-11.3F, 2001]. The enemy agents are placed a bit more randomly 
within the urban area. At the start of a run the enemy agents will meander around toward 
set waypoints until notified of the blue convoy approaching. 

6. Squad Personalities 

In Figure 12 the personality properties tab is displayed. The personality tab is the 
primary area for establishing an agents’ propensity towards certain actions. The 
personality settings available for manipulations are identified in four categories; agent 
situational awareness, squad situational awareness, inorganic situational awareness and 
move constraint. Under the Agent SA settings, the user may define how the agent as an 
individual performs. In the Squad SA area, personality weightings can be established for 
how the squad acts as a unit. In the Jnorganic SA area weightings are established 
determining how agents act when information is received from other friendly agents 
outside of their squad. The Move Constraint settings do exactly that, they set constraints 


on the agents movement based on a user defined criteria. 


The agent situational awareness personality weightings determine how an agent 
will respond to other entities appearing on the SA map. Weightings can be varied 
between -100 and 100. The higher the weighting value, the greater the desire an agent 
has towards the respective action. Alternately, the more negative a weighting the more 
an agent is repulsed. For example, if a value of 100 is placed in the easy going box, an 


agent will strongly desire to stay on terrain which allows for easier movement. 


The squad situational values are parameters, which can be altered based on 
information held by an agent’s parent squad. An agent can display personality traits 
based on what another agent within the squad detects. For example, if the enemy threat 1 
box was given a setting of 10, then if any other agent within the squad detected an enemy 
with a threat of 1, the whole squad would have a desire to go towards that enemy 


detection. 


The inorganic situational awareness parameter settings are similar to the squad 
SA settings. Detections made by all friendly agents in the scenario are placed on the 
inorganic map through a user created communications link. The user can then specify 


parameter settings for an agent’s reaction to information received inorganically. 
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Figure 12. 


Personality Setting Tab. 


Cancel | 


The move constraint set of parameters are designed to affect movements based on 
a particular set of criteria. There are three selections for the user; combat, cluster and 
advance. The combat constraint allows the user to ensure that agents will not advance on 
The cluster constraint allows the user to 


The 


an enemy without a numerical advantage. 
prevent clusters of friendly agents from building up beyond a specified number. 
advance constraint is intended to prevent squads from advancing towards their next goal 


without a specified number of friendly agents within sensor range. 
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For most of the personality settings there is a selection for minimum application 
distance (Min App.) and maximum influence distance (Max App.). These selections are 
designed to set upper and lower bounds on the distance for which entities within that 


distance are included in the calculation for the associated personality weighting. 


Also within this tab, shown in Figure 13, is the trigger state selections panel. A 
state can be selected and then appropriate personality weightings altered as desired by the 
user. For example, if the Shot At(Pri) state is checked, the user might want an agent to 
seek cover and concealment rather than stay on the road when fired upon. There are 
multiple trigger states available, many of which are based on how an agent is informed of 


an occurrence. 
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Figure 13. Trigger States. 
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In our scenario, blue agents begin a run with the desire to move toward the next 
waypoint while staying on easy going terrain and staying within an established distance 
of uninjured friendly agents. Trigger states are used extensively in our scenario. 
Logistical vehicles represented by the truck icon will move along the route until itself, 
their squad or another friend encounters enemy contact. If any of these state changes take 
place, the logistical vehicles will be kicked into the spare J state where their personalities 
can be varied along with the time they remain in the state. Appendix A covers the 
weightings that are varied along with a detailed description of each factor. Blue security 
vehicles have similar personality weightings at the start of each run and utilize the trigger 
states in the same manner as the logistical vehicles. Again, Appendix A covers the 
personality weightings, which are varied for the security vehicles. The cluster constraint 
along with minimum application and maximum application distances of the uninjured 
friends weightings are utilized to force the convoy to travel along the route in positions 
relative to where they started. This option seems reasonable, as it is rare that vehicles 


would pass each other while traveling under normal circumstances. 


Enemy ambush agents are established to begin a run desiring to meander toward 
established waypoints near the convoy route. It is assumed that the enemy will have a 
point of ambush designated and move towards it when notified of a convoy approaching. 
The enemy observer is placed in an area where he will always detect the convoy on the 
route, it is assumed that the enemy scout will locate itself on a vantage point where there 
is observation of the relevant route. Once the convoy is detected, ambushers are notified 
and are kicked into a state where they desire to locate and fire on the convoy. Located 
along the route is an IED, which is armed by the enemy observer. The IED will detonate 


at random point along the convoy. 


The last two blue agents to cover are the UAV and the observer killer squad. A 
UAV will fly along the main supply route (MSR) in front of the convoy. The UAV’s 
goal is to detect enemy agents and alert the convoy, kicking them into an enemy contact 
state. Characteristics of the UAV such as speed and detection capabilities are varied in 
the data farming environment. The observer killer squad has one mission, to fire on the 
red observer agent if the UAV detects the observer. It is assumed that an enemy scout or 


observer would be difficult to detect in the “real world,” to model this, the red observer 
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possesses a high stealth value and will rarely be detected. In the event he is detected, we 
needed a way to simulate the observer being neutralized by either a reaction team, air 
attack or some other means. In an attempt to keep the model as simple as possible an 
indirect fire weapon (observer killer) was set up to fire only on the red observer and only 
if he is detected by the UAV. 

7 Ranges Tab 

Similar to the personality properties tab, the ranges tab is dependent on a squad’s 
current state. Figure 14 displays the ranges tab. The ranges tab allows for general 
settings such as the icon representing an agent within a squad, the allegiance and threat 
level of a squad and the squad’s movement speed in the selected state. The enemy 
interactions selections allow for setting the number of hits to kill an agent, the stealth and 
armor thickness of an agent. The sensor capabilities selections simply control the agents 


sensing characteristics and will be further explained below. 
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Figure 14. Ranges Tab. 
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Under the General settings, the user may establish icons for the squads, their 
allegiance (blue, red, neutral), their threat (low, medium, high), agent class and 
movement speed. Threat pertains to how the selected squad responds to an enemy agent, 
this option can be used to have agents react differently to altering threat levels. 
Movement speed refers to how many cells an agent may move in a given time step. In 
order to have agents move at speeds relative to each other, the blue convoy vehicles were 
set as baseline. We assumed that vehicles in the scenario, under normal driving 
conditions, will move roughly four times the speed of the foot mobile ambushers. The 
speed of convoy vehicles when contact occurs is varied because vehicles have the 


capability to move considerably faster than foot mobile agents at their top speed. 


Moving on to the Enemy Interaction settings, the user may set the number of hits 
required to kill an agent, the stealth of an agent and the armor thickness. In our scenario, 
the number of hits to kill is varied to represent altering degrees of armor. Stealth was 
used for the red agents; it is assumed that ambushers will conceal themselves until ready 
to attack. The convoy agents do not have any stealth, as trucks moving down a road are 


easily detected. 


The sensor capabilities settings are used to establish how many cells away (i.e. 
distance) an agent can detect a contact and also how many cells within that detection 
range can an agent classify a contact as enemy, friendly or neutral. The user may either 
lock the two ranges together or specify probabilities for classifying agents. In our 
scenario the ambusher’s ranges are set larger than the convoy’s due to the fact that the 
enemy will assume positions where they can observe the MSR. The convoy’s detections 


ranges are varied in the scenario to uncover how much it impacts outcomes. 


The Fuel settings were not used in this scenario. The author believes that further 
investigation could be made to see how much fuel consumption might play in longer 


convoy evolutions, where time of mission completion is a key measure of effectiveness. 


The Miscellaneous setting of waypoint radius is simply how many cells an agent 
must be within a waypoint in order to consider it reached. The value was set at twenty 


for all agents to ensure that agents reach and move on to the next waypoint. 
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8. Weapons Tab 

The weapons tab depicted in Figure 15 generally allows the user to specify how 
many and what type of weapons an agent possesses. Under each weapon selected the 
user can specify kill probabilities based on range, the shot radius and establish constraints 
on the number of rounds, rates of fire, penetration of the round and what type of targets 


the weapon can be used against. Again, weapons are dependent on the state of the squad. 
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Figure 15. Weapons Tab. 


A squad may have up to four different weapons. Weapons may be a kinetic 
energy type, such as rifles and machineguns, or be high explosive, such as mortars and 


artillery. Parameters for weapons may be varied in altering trigger states. The primary 
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difference in the kinetic energy and high explosive weapons is that when setting ranges 
and probabilities of kill (PK), the range and PK for kinetic energy is set from shooter to 


target, where the high explosive range and PK is a radius from the point of round impact. 


In our scenario there are four types of weapons in use; however, individual agents 
only possess one type of weapon each. Logistical vehicles possess a kinetic energy 
weapon representing an M-16. The ranges and PKs for the trucks are extremely limited 
and will rarely hit a target. The reason is that the accuracy of an M-16 shot from a 
moving vehicle is extremely inaccurate, but does provide suppressive fire. The security 
vehicles represent armored High Mobility Multipurpose Wheeled Vehicles (HMMWVs) 
with mounted M240G machineguns. The reader is advised that this is not a physics- 
based model to simulate the effects of weapon systems. For this reason, military 
experience and judgment was used to establish the ranges and PKs for the weapon 
systems based on their general characteristics and relation to one another, similar to how 
the movement speeds were established. To further model the ranges and PKs of the 
weapon systems we utilized, an agent-based simulation document from TRAC-Monterey 
was used, where ranges were established for a similar MANA scenario [Brown and 
Cioppa, 2003]. M-16s are less accurate and effective than the ambushers AK-47s and the 
M240Gs have a greater range, rate of fire and effectiveness than AK-47s. The red RPGs 
have a limited range but more explosive power. Since an RPG is a direct fire weapon but 
has an explosive round, the shot radius setting was used to model the destructive power 
of the round. Finally, the IED in the scenario was created using the high explosive type 
weapon, the kill radius was set based on observed explosive power of this type of 
weapon. The reader should be made aware that this is not a model designed to explore 
specific weapons and PKs. That being said, weapons were primarily modeled based on 
their relative characteristics to other direct fire weapons. 

9. Situational Awareness (SA) Tab 

The squad SA tab displayed in Figure 16 is used to establish characteristics of 
communication within a squad. Options on this panel do not vary with trigger state. The 
author recommends that the MANA developers explore ways to implement 
communications based on trigger state. It seems realistic to think that communications 
could be affected by such things as enemy contact. The intra-squad comms delay setting 


at 


enables the user to establish a delay in time steps, between the time a message is sent and 
when it is posted on the SA map. The squad threat persistence option enables the user to 
establish how many time steps a contact will remain on the SA map. The fuse time and 


fuse radius settings are used to classify unknown contacts on the SA map. 


In our scenario, communication limitations are not modeled, all agents are 
assumed to have good communications that do not vary. Though this is far from reality, 
the goal of this thesis is not to identify the impact of bad communications. This is an 
excellent area for follow on research, given that not all vehicles within a convoy will 
have communication assets. “Challenges to convoy movement control included 


inadequate communications” [EFCAT, 2003]. 
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Figure 16. Squad Situational Awareness Tab. 


10. Inorganic Situational Awareness 

The inorganic SA map shown in Figure 17 is similar to the SA map, the 
difference being that there is no “Show Squad Friends” option. This is removed due the 
fact that an agent or squad receiving third party contact information cannot confirm that a 
contact is a squad member [Anderson et al., 2004]. Again, in our scenario, perfect 
communications is assumed, for this reason we will only briefly cover the options 


available on this panel. 


The inorganic panel enables the user to control information from established 
communication links between squads. The Inbound Inorganic Information options 
provide the following options for how incoming information is handled. The min link 


rank accepted offers the ability to determine what information level (low, medium, high) 
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of information is pasted on the squads inorganic SA map. This option could be used to 
model trust issues or establish hierarchical levels of communication networks. The 
inorganic threat persistence allows the user to establish the number of time steps that a 


contact remains on the inorganic map. 


The Outbound Communication Links allows the user to establish communication 
links between the desired squads. In our scenario all blue squads are linked and all red 
squads are linked to the appropriate allegiance. The user has the ability to set up ranges 
of communications, capacity of messages a squad can handle, number of time steps 
required for a message to be sent and the priority level of a message sent. There are 
numerous options available for communications to be varied and manipulated to observe 


the affects of degraded communications. 
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Figure 17. Inorganic SA Tab. 
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11. Movement Algorithm Tab 

Finally, the algorithm tab depicted in Figure 18 provides the user the ability to 
change movement algorithm options. Options on the tab do not vary with squad state. 
The user can specify the type of algorithm used when agents move and establish the 
associated amount of precision when conducting a movement. Precision can be set to 
increase or decrease the amount of randomness in an agent’s movement. Also on this tab 
are the general settings of multiple agents in a cell, diagonal movement correction, how 
obstacles are navigated, whether a squad moves together or not and, finally, whether the 


terrain affects the speed. 


Our scenario uses the Stephen Algorithm, which is identified as the standard 
algorithm by the user’s manual. Blue agents are set with low movement precisions, 
meaning there is little randomness in their selected moves. The idea is that a convoy has 
a set route that it will follow and not meander around towards waypoints. Red agents on 
the other hand were set with higher move precision, allowing them to have some degree 


of randomness when selecting a move. 


Under the general movement settings, the user can turn on or off several options. 
The multiple agents allowed in cell option allow for more than one agent in a cell if 
checked, depending on the “real world” scale, this option might want to be checked. The 
diagonal movement correction option corrects for the time it takes an agent to move if 
they are moving in a diagonal direction. The navigate obstacles option allows the agents 
to move around solid terrain objects if they get “stuck.” The squad moves together 
option ensures that each squad member’s fractional movement is the same at each time 
step. Finally, the going affects speed option will cause the type of terrain to affect the 


speed of agents. 


In our scenario, for the blue agents all options are checked. For red agents, the 
going affects speed option is not checked. The assumption is made that ambushers on 
foot in a familiar area will have the ability to move somewhat freely with terrain not 
affecting their movement. This also allows for modeling the red agents moving in and 
out of structures, though this does not affect their speed, it does affect the ability of the 


convoy to detect them. 


40 


General | Map | Personality | Ranges | Weapons | Squad SA| Inorganic SA Algorithm | Deprecated | 





Precision v 


Precision Move Selection 


GAA oii Pairs iets Move Precision: 5 = 


wil piigh ce 1: | 


Tau SoftMax Move Selection 
we jo: | 
Tau: [0.001 





Pat Following Parameters 


MaxePatibosss [10 3] 
May StiTsP Targets: 3] 
WPinfinence Dist: fo 3] 
Rejection Lock timer fo aii 


Jv linmtResponse to track 
JV TSP Overnides Personality, 
Iv Use trie Sh Target Posi 








i = peering oft2 Default State Cancel | 


Figure 18. Algorithm Tab. 


This concludes the discussion of the chosen modeling environment and the 
creation and characteristics of the created MANA scenario. The model was created to 
identify general properties of a ground convoy that determine their success in this 
scenario. There are hundreds of other parameters within this model that could be 
explored, such as communications, weapons types, ranges and PKs and the introduction 
of a more urban terrain. The reader is advised that all conclusions are based on the 
outcomes from the created scenario and could be significantly different given a different 


set of parameters. 
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IV. ANALYSIS METHODOLOGY 


This chapter presents the analysis methodology for our ground convoy simulation 
model. First, we will cover our selected measure of effectiveness (MOE). Second, we 
cover the experimental design process. Third, we cover the selected statistical software 
package including some of the data manipulations. Finally, we cover some of the 
statistical techniques used to select pertinent factors and analyze the effects. 

A. MEASURE OF EFFECTIVENESS (MOE) 

MOEs are quantitative measures that give insight into the performance of a 
measured activity. In our scenario, we chose blue casualties as the MOE to determine 
how our convoy is performing in the scenario. This MOE was chosen based on 
conversations with individuals involved with I MEF operations. It was stated that during 
the offensive phase of OIF, “time to mission accomplishment was a critical element.” If 
our scenario was based on movements during an offensive phase, we would probably 
have selected time it takes the convoy to reach its goal as a possible MOE. However, our 
scenario is based on the stabilization phase of operations where time is less important and 
survivability of our troops is the priority. 

B. DESIGN OF EXPERIMENT 

In this section we will cover three topics relating to the experimental design. 
First, we will discuss the factors and how they were selected. Second, we will cover the 
experimental design used to generate the factor combination settings. Finally, we will 


discuss the data farming process for obtaining the dataset used in the analysis. 


Utilizing trigger states, MANA offers the ability to examine thousands of possible 
factors within a given scenario. Obviously there are computing constraints and it would 
be impossible to examine every possible factor within a scenario in a single design. 
Since the purpose of this thesis is to examine the effects of actions when an ambush is 
encountered, the majority of the examined factors occur in one state. In order to 
understand where in the scenario a specific factor occurs and which squad it affects, they 
will be identified by the squad, then state and finally factor in the following organization 
squad_name/state_name/factor name [Wolf, 2003]. To illustrate the factor for a security 


squad’s aggressiveness, the factor will be written as, BlueSecurityFront/Spare 1/Enemies. 
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1; Factors and Ranges 

Realizing the difficulty in examining every possible factor and combination of 
factors in a single MANA scenario, we identified the factors that logically should be 
important, based on experience and a literature research, such as placement of security 
vehicles within a convoy. Once a list of possible factors was compiled, we began to 
make exploratory runs in an attempt to make a preliminary assessment on the 
significance and to bound the ranges of those factors. Though there are literally 
thousands of possible factors and combinations within a single scenario, not all the 
factors need to be varied in every state. For example, in our scenario, we are interested in 
what happens after a convoy makes enemy contact, therefore, many of the factors were 
irrelevant in states other than the contact states. However, there are factors that are 
examined outside of the contact states, such as security vehicle locations, where the state 


does not affect the factor. 


Once a baseline scenario was created, we began to zero in on those factors that 
logically should be significant. We began our experiment by conducting a 22 factor 129 
level design. The 22 factor 129 level design refers to the Latin Hypercube technique of 
experimental designs and is discussed further in the next section. The results of this 
design displayed several factors that were expected to be significant, such as security 
vehicle location, security vehicle aggressiveness and logistical vehicle speed. After 
doing regression analysis on the results, we were able to identify 16 main factors 
dominating the model. Next, we ran a 16 factor, 65 level design to further zero in on the 
important factors. From these results, we were able to narrow the design down to 11 


important factors. 


Upon examination of our output from the final 11 factor production runs we 
found some disturbing trends. Initially it appeared that the means of each excursion were 
identical only differing between the scenario used for the excursion. We utilized four 
scenarios to vary the location of the security vehicles within the convoy. Upon further 
investigation it was discovered that the output for each excursion was identical in mean, 
standard deviation and range. We expected the means between excursions to be 
relatively close due to the fact there are only 16 blue agents. However, the fact that the 


distributions of all 129 excursions with extremely different parameter settings were 
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identical was realized to be impossible and far too coincidental. After examining the 
scenario and experimental design to ensure we had not made an error, we turned to our 
support at the MITRE Corporation who had conducted the simulation runs. Upon further 
examination it was found that there was an error in the scripting of the simulation runs. It 
should be noted that the discovery of the problem and correction of the error took only 
two days. While this might seem to be a major set-back, in reality it seems to validate the 
power of ABMs and our chosen experimental design. The ability to produce numerous 
simulation runs and examine the data quickly for unlikely trends reinforces the value of 
ABMs and the data farming technique. Consider, for example, a large physics-based 
model that takes weeks to set-up and then 60 hours to run. If a similar problem had 
occurred, weeks of work would have been lost. In our case this was a relatively non- 


issue and caused only a minor set-back. 


Once the error was discovered we were concerned that the error had existed in our 
previous simulation runs. The error was discovered to be present in our 22 and 16 factor 
design runs. In order to avoid taking a step back in our research we went back to 
previous 11 and 16 factor exploratory runs where the error was not present. These data 
sets, along with experience and research, were then used to select our factors for the final 


11 factor production run. 


The factors listed in Table 2 are the factors used in our final production runs. The 
factors are listed by affected squad or squads, the state of the factor, the actual factor, 
plus the low and high level of the settings. Recalling from Chapter III, there is one 
security vehicle in each of the security squads. Factors that affect the security vehicles 
are varied together, meaning for all practical purposes they act as a single squad with 


equal desires, speeds and so forth. 
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Squad Name Factor Name Low Setting High Setting 
Security Vehicles Start of Run Security Location 
Spare | Sec Speed 

Sec Time in State 

Sec AgentSA Enemy 
Sec InorgSA Enemy 
Sec AgentSA Friends 
Default Sec Detect Range 
Logistic Vehicles Log Time in State 
Lod Twd concealment 
Log Detect Range 
Log Speed 

Default Detect Range 












































Table 2. Selected Squad/State/Factors. 


To relate the speed and distance factor settings to the “real world,” the following 
is provided. Recall from Chapter III that the speed of the vehicles was set as a baseline, 
meaning all other agent speeds are relative to vehicles. A speed of 100 means a vehicle 
moves | pixel per time step, so if we assumed that a convoy was moving 30 mph then a 
person on foot would move roughly one-third of this or 10 mph at the maximum. Agents 
on foot were set up to have a speed a bit faster than would be in reality in order to model 
their knowledge of the terrain. Continuing with the example, if the security vehicle speed 
is set at 250, they can move 2.5 pixels per time step and implies they can move roughly 8 
times the foot soldier or 80 mph. Eighty mph is extreme but was purposely set this high 
to observe any extreme effects or points of diminishing return. Detection range distances 
were purposely set low for the convoy in order to simulate the effects of moving vehicles 
in a column and to give a slight observation advantage to the ambushers. For example, if 
we assumed that each pixel is 3 meters then an agent’s detection range would be 30 
meters and if 100 then their range 300 meters. 

2. Nearly Orthogonal Latin Hypercube (NOLHC) 

Current and foreseeable future computing capabilities eliminate the possibility of 
attempting a full factorial experiment with numerous factors containing expanded levels. 
Suppose we wished to examine our 16 factors with each factor having 3 levels. Using 3 
levels would enable the discovery of any non-linearities within the model. To create a 
design of this nature we would need 3'° design points, which would require 43,046,721 


runs to get a single data point for each combination of factors. Now suppose that each 
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run takes 30 seconds, this would require 21,523,361 minutes, 14,947 days or 
approximately 41 years to get a single data point for each design point. Obviously, a full 
factorial design was unacceptable; especially considering that the factors selected all have 


more than 3 levels. 


Considering that our factors all contain more than 3 levels, a full factorial design 
was immediately ruled out as a possibility for a design; so we pursued a Latin Hypercube 
(LHC) design. The LHC design is a sampling technique where the goal is to sample the 
experimental area not only on the edges but also in the interior to maximize the space- 
filling of the sampled area [Cioppa, 2002]. Using an LHC design minimizes the 
assumptions that need to be made about the form of the response surface. The NOLHC 
design provides us with two desirable characteristics for examining our scenario; 
approximate orthogonality and good space filling. The LHC technique provides an 
efficient method to examine many factors each with multiple levels in an attempt to 
uncover any non-linearities within the response surface [Brown et. al., 2002]. To 
construct the LHC for this thesis, we used a Microsoft Excel spreadsheet created by 
Professor Susan Sanchez [Sanchez, 2004]. The spreadsheet displayed in Figure 19 
allows the user to first pick either an orthogonal or nearly orthogonal design based on the 
number of factors required. The user specifies the low and high levels of each factor, the 
algorithm then divides the range into subpopulations of equal marginal probabilities. The 
algorithm then takes a random sample from one of the subpopulations and sets this value 
as the first factor setting for that particular design point. The process is repeated until all 
levels of the factor are filled. The resulting design is a matrix of randomly sampled, 
randomly assigned and uniformly distributed factor settings [TRAC-Monterey, 2003, 
Cioppa, 2002 and Wolf, 2003]. 


When using a NOLHC design, a major concern is correlation between the factor 
columns or the existence of multicollinearity. The process for constructing the design 
itself limits the amount of correlation present. However, in an attempt to gain more space 
filling and further reduction of possible correlations, we linked 4 NOLHCs together. 
This was done by randomly shuffling the 11 factor columns, placing the lower and upper 
level values in the excel spreadsheet and creating new design points. After the new 


design points were created, the columns were placed back in the correct factor order and 
47 


concatenated with the preceding design. In order to gain more observations and space 
filling, we then took the design and ran each of our four scenarios with the entire design. 
The resulting design after linking was a 11 x 129 matrix with no correlations greater than 
0.02 or less than -0.02. Figure 20 displays the scatter plot and Figure 21 displays the 
associated correlations for each pair of factors. One of the main factors we wanted to 
examine was the security vehicle location; this factor was removed from the matrix and 
replaced with another relevant factor when the actual runs were made, we were careful to 
ensure that the MANA scenario that this factor corresponded to was visible in the data 
output. When the run results were obtained, the factor was placed back in the output 


based on which scenario was used with the corresponding output. 
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Figure 19. LHC Design Spreadsheet (From: Sanchez, 2004). 
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Figure 20. Scatter Plot. 
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Figure 21. Pairwise Correlations. 
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3. Computing Process 

The MITRE Corporation is a defense contractor that maintains an office in 
Woodbridge, VA, supporting PA and other Marine Corps offices. MITRE operates a 
supercomputing cluster that supports data farming efforts. Once a scenario and design 
are created and factors are ready to be data farmed, the files are simply e-mailed to 
MITRE. MITRE then creates a script file that establishes which factors in the 
appropriate trigger states are to be varied. The user specifies how many replications are 
to be performed at each design point; when the runs are conducted a random seed is 
applied to each row. Once the runs are completed the output files are returned in the 
form of a comma separated value (.csv) file. For our final scenario production runs we 
had 129 design points and conducted 50 runs at each point, this requires 6,450 runs. In 
order to examine the security vehicle location factor this process was conducted 4 times 
for a total 25,800 runs. 
C. ANALYSIS OF THE DATA 

There are numerous software packages available for data analysis, such as S-Plus, 
SAS, Clementine and JMP. The author was introduced to the JMP Statistical Discovery 
Software '™ during this thesis and found its user-friendly construction desirable. For this 
reason it was the package used for this thesis. In this section we will briefly cover the 
chosen data analysis software package and discuss the manipulation of the simulation 
results. 

1. Chosen Data Analysis Software 

JMP was created by the SAS Institute Inc. and possesses many of the desirable 
attributes of several other software packages rolled into one. “JMP is designed to be a 
point-and-click, walk-up-and-use product that harnesses the power of interactive 
statistical graphics to serve the analysis needs of the researcher” [JMP, 2002]. The author 
was able to learn the main capabilities of the package within a day and the extensive 
graphical user interface (GUI) capabilities make the manipulation of data extremely 
simple. JMP allows the user to view and manipulate data in a spreadsheet, produce 
subsets of data and provides a wide range of possible graphical, statistical and data 


analysis options [JMP, 2002 and Wolf, 2003]. 
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2. Data Consolidation 

The simulation output data consisted of 25,800 observations. When the data was 
returned each design point excursion is represented by a unique identifier code. Each 
identifier consisted of 50 observations. In an attempt to identify whether the factor 
settings or the variability within the simulation were causing particular outcomes, the 
data was grouped by identifier code and the mean value of the response variable (blue 
casualties) was taken for the 50 points in each excursion. This resulted in 516 data points 
representing the average of each particular excursion. Before the consolidation was made 
we added our 4 security vehicle locations back into the data. This was accomplished by 
adding a column for each location. The values of the columns were either 1 or 0 
representing which location the row of settings corresponds to. The resulting matrix after 
this manipulation was 15 x 129. 
D. ADDITIVE MULTIPLE REGRESSION 

In an attempt to fully understand what factors in our scenario were contributing 
the greatest to the success of our convoy in the created scenario, we applied the additive 


multiple regression analysis technique to our data. 


Regression is statistical technique that investigates the relationship between two 
or more predictor variables to a response variable [Devore, 2000]. For example, if we 
found that convoy speed was a statistically significant predictor variable, we could 
perform regression analysis to determine how many blue causalities we would expect to 
suffer based on speed. In our model, we only considered main effects, two-way 
interactions and 2™ degree polynomials. 

1. Regression 

Relatively recent advances in computing power have enabled multiple regression 
analysis to become a widely used technique by researchers. Linear regression fits a linear 
function of predictor variables to a continuous response variable using the least squares 
fitting criterion [Devore, 2000 and Hamilton, 1992]. The general additive multiple 


regression model equation is listed below: 


Y=£,+/x,+ 6x, +...+ Bx, +6 
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Y is our response variable; x is the independent or predictor variables, is the 
coefficient of the predictor variables and ¢ is the error term. 

2. Regression Assumptions 

In order to apply the statistical testing procedures of t-tests and f-tests there are a 
few assumptions that must hold regarding the error term in order for the tests to be valid. 
Errors or residuals must follow a normal distribution, residuals must be identically 
distributed with zero mean and constant variance and all residuals must be independent 
and identically distributed [Devore 2000, Hamilton 1992 and Wolf 2003]. 

3. Model Comparison 

In our comparison of the regression models created from our data, we use the 
coefficient of determination (R’) value to determine which model is providing the best fit 
to the data. The R’ value can be interpreted as the proportion of observed variation in the 
response variable that can be explained by the linear regression model [Devore, 2000]. 


The equation to calculate the R’ value is provided below: 


SSE 


ae, Peace 
SST 
Where SSE is the sum of squared deviation about the least squares line and SST is 


the sum of squared deviations about the mean response line. 


An R° of 1.0 means that the model completely explains the response variable 
based on the predictor variables. Alternately, an R° of 0.0 indicates that there is no 
relationship between the predictor variables and the response variable [Wolf, 2003]. 
Although, we looked for high R’ values in our models, there are other considerations to 
be made when comparing the models. Also, in multiple linear regression it is possible for 
the R° value to provide a deceptive measure, it can become greatly inflated when a large 
number of predictors are used relative to the number of observations [Devore, 2003]. 
With 11 main factors we could have a full model with 78 terms consisting of main 
effects, two-way interactions and polynomial terms to the oe degree. With 78 predictor 
variables it would be extremely difficult to determine what is actually happening in the 
model. For this reason we attempted to minimize the number of predictor variables in 


model while still maintaining a reasonably high R’ value. 
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We used an F-test to further evaluate prospective models. The F-statistics allow 
us to test hypotheses regarding sets of parameters [Hamilton, 1992]. Given a large 
complex model we can then take a subset of parameters from the large model and test 
whether the large model significantly improves upon the smaller model with fewer 
parameters [Hamilton, 1992]. Suppose we have K predictors in our model minus the 
intercept term and wish to reduce the model by H predictors, we can then observe 
whether the latter model significantly improves from the lesser. The relevant equation is 


listed below [Hamilton, 1992]: 





fig (RSS{K — H}— RSS{K})/H 
ae: (RSS{K}) (n—K) 
Where RSS{K} denotes the residual sum of squares for the larger model (K 
parameters) and RSS {K-H} is the residual sum of squares for the model with fewer 
parameters. The calculated F statistics are then compared to the F-distribution with df; = 


H and dfz = n-K degrees of freedom. 


In summary, the F-test compares the null hypothesis that all predictor coefficients 
in the model equal zero against the alternative that at least one of the predictors is not 
zero and therefore provides some useful information about prediction of the response 
variable. 

4. Significant Terms 

In order to determine which factors in the models were providing the most useful 
information we needed to test the individual terms of the models. Where the F-test 
allowed us to compare models, the f-tests allows us to compare individual predictor terms 
in the model. We used the ¢-test to accomplish this task. The t-statistic test allows us to 
evaluate the importance of the individual coefficient values. The t-statistic is calculated 


with the following equation [Hamilton, 1992]: 
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Where b, is the estimation of the coefficient for parameter k, 7, is the actual or 
hypothesized value of the coefficient of parameter k and SE,, is the estimated standard 
error of b;, the estimated coefficient. 

Each t-statistic of the predictors equals the coefficient divided by its standard 
error. The corresponding p-value for each coefficient is also provided with most 


statistical software packages. A p-value greater than .05 indicates that a term is not 


significantly significant [Devore, 2000]. 


In conclusion, we have identified our experimental design methodology, 
identified the process for selecting the factors and briefly explained the additive multiple 


regression technique. Chapter V will detail the analysis conducted on our dataset. 
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V. DATA ANALYSIS 


In this chapter, we detail our analysis findings. We begin with an initial 
assessment of the data. Next, we cover the multiple linear regression models with main 
effects, and then examine the two-way interactions and the quadratic model. We will 
combine all relevant terms into a final regression model and summarize the selected 
factors. Finally, in each section we explore what the resulting factors suggest about our 
scenario and how they might relate to the “real world.” 

A. INITIAL ASSESSMENT OF THE DATA 

We begin our analysis by doing an initial assessment of the data where we look 
for any obvious errors, the distributions of blue and red casualties and any obvious trends 
and outliers. It should be noted that the data farming process does take some manual 
effort that can possibly lend itself to errors. Mentioned in Chapter IV was an example of 


some processing errors we found in several of our datasets. 


Upon receiving the data we created a distribution plot of the blue and red 
casualties with associated summary statistics, see Figure 22. Particular attention should 
be paid to the high number of runs that resulted in 12 blue casualties, we will examine 


this occurrence later this chapter. 


Distributions 
Alleg1Cas(Blue) Alleg2Cas(Red) 


Moments Moments 
Mean 6 2403486 5.5883721 
Std Dev 3.4130066 2.2849672 
Std Err Mean 0.0212485 0.0142256 
upper 95% Mean 6.281997 upper 95% Mean 5.616255 
lower 95% Mean 6.1987007 lower 95% Mean 5.5604892 
nN 25800 nN 25800 
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Figure 22. Blue and Red Casualty Distributions with Summary Statistics. 
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We then plotted the data on a scatter plot to look for any obvious trends and 
outliers. Figure 23 displays the scatter plot and highlights some obvious outliers that will 


be investigated later in this chapter. 
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Figure 23. Scatter Plot with Highlighted Outliers. All rows on the Y axis 
corresponds to each design point in the data set. 


After doing an initial assessment of the data, it is clear that there are some 
interesting effects taking place. In the next section, we cover our regression analysis. 
B. FITTING THE MODELS 

Our final production run consisted of 11 factors believed to be logically relevant 
in our scenario based on military experience, judgment and the results from exploratory 
runs. Security vehicle location is our 12" factor; locations were modeled using 4 separate 
scenarios and are used in the regressions. In this section, we detail the regression models 
created and relevant factors identified in each model. We then combine the dominating 
terms into a final model and closely examine the main effects and two-way interactions. 
Finally, we summarize some of the interpretations of the factors and interesting outliers. 

1. Main Effects 

We began the regression analysis with a model consisting of only the 12 main 
effects. There are actually 15 terms in the initial regression model, 4 of which represent 


the security vehicle locations. We used the mixed stepwise regression function in the 
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JMP software package to conduct the regression. The mixed function alternates the 
forward and backward steps, entering terms based on a specified significance level in the 
forward step and then removing terms with the least significance in the backward step. 
The resulting model contains all significant terms [JMP, 2002]. A term is initially 
included in the model if its p-value was less than 0.25 for a forward step. We set the 
criteria for a term to be removed from the model in the backward step if the p-value is 
greater than 0.05. A p-value less than 0.05 is generally considered to be significant 


[Devore, 2000]. 


The main effects model started with 15 terms and the initial step through resulted 
in 10 terms being significant with an R° of 0.4811. Figure 24 displays the actual value by 
the predicted value plot. The general trend of the data is positive indicating that the 
actual values of the blue casualties and predicted values of blue casualties are in general 
agreement. Annotated by the circles are some obvious groups of outliers that will be 


further examined in the analysis. 
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Figure 24. Actual versus Predicted Blue Casualties. 
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Before we move on to adding first order interactions we wanted to examine some 
aspects of the model a bit more closely. In previous exploratory runs security vehicle 
locations had dominated the models. In this model, only one location term entered in the 
model is significant. Security location 4, which was purposely created as an extreme case 
that no convoy commander in their right mind would utilize, enters as a significant term. 
Upon closer examination, the term has a positive slope indicating that if this location is 
used the convoy experiences more casualties. This was expected and reassured us that 
the scenario was performing as we expected. At this point, rather than eliminate any of 
the factors that only slightly effect the model we decided to leave them all in the model 
and try adding first order interactions in an attempt to improve the fit (i.e. increase the R? 
value). 

Ze First Order Interactions Model 

Rather than start the stepwise function with only the significant terms from the 
main effects model, we decided to include all possible interactions. After the initial 
screening, the resulting model consists of 33 significant terms with an R° of 0.7993. Now 
we have 12 main effects and 21 interaction terms. Even though the explained variance in 
this model is much larger than the main effects model, it would be extremely difficult to 
understand what was happening with all 33 terms. Taking a closer look at the included 
terms it is evident that there are only a handful of terms that dominate the model. If we 
only use 3 terms; the security inorganic aggressiveness, the logistics vehicles propensity 
to move toward concealment and the interaction of the two, we can explain 52 percent of 
the variation in the model. Figure 25 displays the leverage plots of these 3 terms. 
Leverage plots give a graphical representation of an effect’s significance test. The 
distance from the points in the graphs to the horizontal line (mean) show what the error 
would be without the term in the model. The distance from the points to the line of fit or 
the diagonal show the actual residual [JMP, 2002]. In a leverage plot we are primarily 
looking to see that the fitted line follows the data better than the horizontal line. 
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Figure 25. Leverage Plots for 3 Dominating Terms. 


There are obviously some critical interactions affecting the model. Rather than 
dig into those interactions at this point in the analysis, we first added quadratic terms and 
then attempted to understand what is taking place. 

3. 2" Order Quadratic Model 

Again, rather than add quadratic terms based solely on the terms from the two 
previous models we decided to start the stepwise function with all main effects, first 
order interactions and second degree polynomials. After running the stepwise function 
the resulting model consisted of 29 significant terms with an R’ of 0.8075. We wanted to 
eliminate terms that were not providing much useful information while still maintaining a 
relatively good explanation of the variance. To accomplish this we reduced the criteria 
for terms to remain in the model by reducing the p-value level to 0.01. Only terms with a 
p-value of 0.01 will remain in the model. By doing this screening, the resulting model 
consists of 23 terms with an R° of 0.7927. Since this screening did not eliminate many 
terms we decided to look at each individual term in an attempt to whittle down the model. 
Viewing the step history produced by JMP, we saw that 3 terms explain 52 percent of the 
model. Figure 26 displays the 25 terms on the X axis and the R° value on the Y axis. 
From this plot, we can plainly see that there is a point of diminishing return. In order to 
increase the R’ value of the model, we added 11 more terms to the three main terms. At 
this point, the model consisted of 14 terms with an R? of 0.7421. We feel that the R? 


value from this model is sufficiently high to be used as our final model. In the next 
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section, we will further eliminate some irrelevant terms and produce the final model. We 
will then examine this model in full detail and attempt to understand exactly what is 


happening in the scenario based on these terms. 
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Figure 26. Model Terms vs. R Squared Value 


Term Parameter 
1 (Sec Inorg SA aggressiveness-0.03101)*(Log twd conceal-0.03101) 
2 Log twd conceal 
3 Sec Inorg SA aggressiveness 
4 Log class/detect range 
5 Sec Time in State 
6 (Log twd conceal-0.03101)*(Log twd conceal-0.03101) 
7 (Sec stay with friends-0.03101)*(Sec stay with friends-0.03101) 
8 (Sec Time in State-66.5039)*(Log twd conceal-0.03101) 
9 Sec stay with friends 
10 (Log class/detect range-55.031)*(UAV class/detect range-65.03 1) 
11 UAV class/detect range 
12 (Sec Time in State-66.5039)*(Sec stay with friends-0.03101) 
13 Sec agent aggressive 
14 Sec Loc4 
15 (Security Speed-175.031)*(Log twd conceal-0.03101) 
16 Log time in state 
17 (Sec stay with friends-0.03101)*(Log time in state-66.5039) 
18 Log Speed 
19 (Security class/detection range Default-55.031)*(Log class/detect range-55.03 1) 
20 (Sec stay with friends-0.03101)*(UAV class/detect range-65.03 1) 
21 (Sec Time in State-66.5039)*(Log Speed-175.031) 
22 (Log Speed-175.031)*(Log Speed-175.031) 
23 (Security Speed-175.031)*(Sec Time in State-66.5039) 
24 Security Speed 


Table 3. Terms Corresponding to Figure 26. 
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4. Final Model 

Before spending too much time looking at each of the 14 terms, we first checked 
the assumptions made. We plotted the actual number of blue casualties against the 
predicted number and the residuals against the predicted number of blue casualties. 


Figure 27 displays the actual vs. the predicted number of blue casualties. 
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Figure 27. Actual vs. Predicted Number of Blue Casualties. 


From the actual vs. the predicted number of blue causalities plot we can see a 
positive slope, indicating that the actual and predicted numbers of blue casualties are in 


general agreement. 


To check the assumption of normality for the residuals we plotted the residuals 
against the predicted number of blue casualties displayed in Figure 28. When viewing 
this plot we want the residuals to have a mean of zero, constant variance and be 
identically distributed. At first glance there appears to be a trend in the data when the 
predicted number of blue casualties is greater than 8. It seems that when above 8 the 
model consistently under predicts the number of blue casualties. This group of points can 
be seen in previous plots when the number of blue causalities is greater than 8. At first 
this was disturbing because it indicates something is happening that is not captured in the 
model. After examining the scenario, it was concluded that this phenomena can 
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attributed to an interaction between the logistic vehicles going towards concealment and 
the security’s propensity towards the enemy when the information is received 


inorganically. This interaction is discussed in the next section. 
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Figure 28. Residual vs. Predicted Number of Blue Casualties. 


The final model consists of 14 total terms, 8 main effects, 4 two-way interactions 
and 2 quadratic terms. In the next section, we examine some of the most influential 
individual terms. 

3: Significant Individual Factors 

In this section, we use one-way analysis of variance tests to get an idea of which 
individual factors have the most significant impact on the number of blue casualties. We 
also identify the factor settings that contribute the most to the response (blue casualties). 
While we examined each of the main effects, we only present those that have the most 
significant impacts. We used t-tests over the entire range of each factor to determine the 


points where the settings are significant. 
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Figure 29 displays the means diamonds plot of logistics vehicles propensity 
towards concealment versus the number blue casualties. The variance at each factor 
setting shows vertically on the plot. The diamond around each column of points indicates 
the 95% confidence level for the predicted number of blue casualties. We notice from the 
plot that when the factor settings are less than 10, the variance in our predictions is much 
less. A negative factor setting means the logistic vehicles want to stay away from 
concealment and on the road moving towards the next waypoint when the convoy is 
ambushed. We can easily relate the setting of 10 to the scenario. In order to keep the 
convoy moving in a column we set the desire to stay on easy going terrain and move 
towards the next waypoint at 10, these factors were not varied in our final production 
runs. When the move toward concealment factor is set at 10, the three factors cancel 
each other out when MANA calculates the movement penalties. When the toward 
concealment factor is set greater than 10 it out weighs the others and the convoy moves 
toward concealment upon contact. When the factor is set less than 10, the convoy desires 
to stay on the road. As the toward concealment factor becomes more negative the desire 
to stay on the route is greater. In summary, in our scenario the convoy does better when 


they do not seek concealment upon contact. This is similar to doctrinal practices. 
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Figure 29. One-Way Analysis of Variance for Log Vehicles toward Concealment and 


Number of Blue Casualties. 


Figure 30 displays the one-way analysis of variance plot for the security vehicles 
inorganic propensity of moving towards the enemy versus the number of blue casualties. 


The relevant factor here means that the security vehicles receive detection information 
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from agents outside of their squad. From the plot we can see that factor settings greater 
than 0 are for the most part not statistically different, however, below 0 the variance 
becomes greater and are significantly different than positive settings. We reason that, as 
the security vehicles receive detection information, the more positive the setting the 
greater the desire to go after the enemy, hence the convoy experiences fewer casualties. 
With this in mind it will be interesting to view the interaction of this factor with the 


logistic vehicles toward concealment. 
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Figure 30. One-Way Analysis of Blue Casualties vs. Security Vehicle Inorganic 
Aggressiveness. 


The last individual factor we will cover is the logistics vehicles detection range in 
the enemy contact state, see Figure 31. We would normally assume that as the detection 
ability of the convoy increases, we should see fewer blue casualties. However, that is not 
what we see in the plot. Several explanations could all be causing the output or 
individually causing. We notice in the plot that there is a general trend, as range gets 
larger, the blue casualties decrease. We also notice that there is a significant amount of 
variability throughout the plot and the mean tends to jump between neighboring settings, 
which we did not expect. After some careful considerations, we concluded that the 
variability could be due to the buildings in the scenario causing line of sight constraints 
on the convoy. If this was true we would expect a point in the range where blue 
casualties rise and stays for points thereafter, which is not the case. At this point we 
turned to the scenario’s construction to see if this could be due to the factor settings. The 


logistics vehicles detection range is varied in the enemy contact state, meaning that once 
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the convoy encounters the enemy, the factor is varied. The reasoning behind this was 
originally to see if a heightened state of awareness once ambushed leads to fewer blue 
casualties. To see if this is true we constructed a contour plot of logistic vehicle detection 
range and security vehicle inorganic aggressiveness against the number of blue casualties, 
see Figure 32. The plot shows an area towards the top right side where casualties are 
lower when the detection ranges are high and security vehicle aggressiveness is high. We 
can conclude that an increased state of awareness during an ambush seems to make a 


difference in our scenario. 
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Figure 31. One-Way Analysis of Logistic Vehicle Classification Range vs. Number 
of Blue Casualties. 
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Figure 32. Contour Plot of Security Vehicle Inorganic Aggressiveness and Logistic 
Vehicle Detection Range vs. Number of Blue Casualties. (Best Viewed in Color) 
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We have identified some of the individual factor effects in our model, next we 
will cover some of the interesting interactions that seem to play a bigger role in our 
scenario than the individual factors. 

6. Significant Interactions 

In our model interactions between factors seem to be explaining a large portion of 
the variance. We use contour plots to examine the interactions more closely. In a 
contour plot, two factors are plotted along the axes with the response variable (blue 


casualties) depicted as a contoured regions within the plot. 


Of the three interactions in our model, the interaction between the security 
vehicles inorganic aggressiveness and the logistics vehicles desire to seek concealment is 
by far the most influential explaining roughly 20 percent of the variance in the model. 
Figure 33, displays this contour plot. Areas of interest are highlighted with ovals and 
numbered 1, 2 and 3. We can plainly see in oval 1 when concealment is greater than 10 
and aggressiveness less than 10, the convoy suffers high casualties. In oval 2, when 
concealment is still high but aggressiveness less than 10 casualties drop to about 5 and 
fewer. In oval 3 where concealment is less than 10 and aggressiveness positive, 
casualties are again relatively low. Relating this to our scenario provides some 
interesting perspectives. It should be noted that the value of 10 can be related to the 
scenario construction, agent personality traits that are not varied were set at 10, so when 
our varied factors are altered, we see differences outside of that value. In some aspects, 
the value of 10 can be thought of as a baseline. In oval 1 our logistics vehicles are 
seeking concealment upon enemy contact while simultaneously the security forces are 
not seeking to engage the enemy; intuitively we are suffering high casualties. In oval 2 
the logistics vehicles are seeking concealment, but now the security forces are engaging 
the enemy, again, intuitively we are suffering fewer casualties. However, and more 
interesting in oval 3, the logistics vehicles seek to continue along the route upon contact 
while the security forces engage the enemy. This in itself is not that interesting but if we 
look closer at the plot, we see that even when the security aggressiveness is negative we 
still suffer roughly the same number of casualties. Although, the convoy does better 


when security vehicle aggressiveness is high, the plot suggests that the actions of the 
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logistic vehicles play a bigger role than does the actions of the security vehicles. This 
occurrence could play a role in training logistic vehicle operators, by emphasizing their 


procedures upon enemy contact. 
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Figure 33. Contour Plot of Security Vehicle Inorganic Aggressiveness Against 
Logistic Vehicle Towards Concealment. Ovals 1, 2 and 3 Represent Areas of Interest. 
(Best Viewed In Color) 


In Figure 34, we examine the interaction between the security vehicles time in 
state and the logistic vehicles’ propensity towards concealment. In our scenario 
construction, the default state acts as a baseline where the logistic vehicles stay on the 
route and keep moving and the security vehicles only slightly desire to engage the enemy. 
Enemy contact state spare I was established to vary the convoy’s tactics upon enemy 
contact. If the time in state is high, there can be several effects, the security will either 
pursue the enemy for an extended period or stay away from the enemy. The scenario was 
purposely designed so we could see the effect of the security vehicles pursuing the enemy 
rather than simply firing and moving on to the next waypoint. In order to understand 


what is happening we need to look at several different plots. In Figure 34, we see that the 
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convoy does the best when the logistic vehicles stay away from concealment regardless 
of the security time in state, as seen in oval 2. In oval 1 we see that the casualties are 
high when time in state is high and the logistic vehicles desire to move toward 
concealment high. This is explained by the fact that the security will pursue a contact for 
an extended period while the logistic vehicles hide; this leaves the convoy vulnerable to 


enemies who have not yet been detected. 
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Figure 34. Contour Plot of Security Vehicle Time in State vs. Logistic Vehicles 
Propensity Towards Concealment. Ovals 1 and 2 Highlight Areas of Interest. (Best 
Viewed in Color) 


To interpret what is happening in the scenario we looked at a contour plot of the 
security inorganic aggressiveness vs. security time in state, see Figure 35. As expected, 
as security aggressiveness increases, casualties decrease regardless of the time in state. 
In addition, as aggressiveness decreases and time in state increases, there are a high 
number of casualties. After careful consideration of the scenario, we conclude that this 
could be caused by the inner workings of the scenario. For example, when time in state 
is low and aggressiveness high, the security will pursue the enemy every time there is a 
detection, which has the same effect as increasing the time of the contact state. This 


explains why we see the same outcome across the range of the time. However, when 
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aggressiveness is low and time in state is high we see the opposite effect. The security 
stays away from contacts and does so for the duration of the respective time. This 
explains why we see high casualties in the lower right corner of the plot. Though this 
might seem intuitive, in our scenario it suggests that the success of the convoy is not 
affected by the security pursuing the enemy or by staying with the convoy and providing 


suppressive fire. 
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Figure 35. Contour Plot of Security Vehicle Time in State vs. Security Vehicle 
Inorganic Aggressiveness. Highlighted Circles Identify Areas of Interest. (Best Viewed 
in Color) 


In this section, we have identified some of the critical interactions in the model. 
In the next section, we will look at some of the outliers that we saw in Figures 23 and 24. 

Te Outliers 

To begin examining the groups of outliers we refer the reader back to Figures 23 
and 24 on pages 54 and 55 respectively. To take a closer look at some of the outliers we 
use a regression tree to see where the main effects of the model make significant splits in 
the data. Viewing Figure 36, we can easily see that our splits occur in our outlying areas. 
The first split occurs at the logistics vehicles toward concealment term, 340 of the 516 


observations are identified when the factor is less 12. When the logistic vehicles stay 
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away from concealment, we see the mean drops to 5.4 blue casualties. When towards 
concealment is greater than or equal to 12 the mean rises to 7.8, and there is obvious 
variance in the data. The next split begins to explain our outliers. When the security 
vehicles’ inorganic aggressiveness is greater than or equal to 2 the mean drops to 5.21 
blue casualties, when less than 2 the mean makes a big jump to 10.9 and dissects the 
group of outliers in the top right of Figure 36. The final split occurs when the logistic 
vehicles detection range is 58 pixels. When greater than or equal to 58 the mean 
casualties are 4.03 and when less than 58 the mean is 6.61. Consequently, these splits 
produce an R’ of 0.828 and explain our three groups of outliers. In our scenario, we can 


conclude that a convoy experiences the greatest success at these factors settings. 
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Figure 36. Regression Tree of Main Effects. 
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8. Interesting Observations and Interpretations 

In this section we look at some of the interesting observations learned while 
examining the data and scenario. We will look at the location of security vehicles within 
the convoy, how the UAV helped in the scenario, the relationship between inorganic 
aggressiveness and the organic aggressiveness and finally how vehicle speeds impact the 


model. 


When we began building the scenario, one of the key factors we wanted to 
examine was the location of security vehicles within the convoy. Though three of the 
four locations did not survive the screening process for the final model, there are some 
interesting observations to be made from the 4 locations. Security location 1 evenly 
distributes security amongst the convoy, location 2 places 2 vehicles in the front and 1 in 
back, location 3 places | in the front and 2 in back and, finally, location 4 places all 3 
security vehicles in the center of convoy. Location 4 is the only placement that made it to 
the final model. As we expected, location 4 has a positive slope, which indicates more 
casualties when used. Although we expected location to play a bigger role in the 
outcome, we are pleased to see that location 4, which would be absurd in real life, 
exhibited this outcome in the results. After observing the scenario and watching the 4 
locations run we concluded that the inability of MANA to constrain agents maneuvering 
amongst each other caused the differing locations to produce similar results. We 
attempted to utilize the cluster constraint and the max and min influence settings to 
inhibit the vehicles when moving near each other, but it did not seem to affect the 
outcome. This could be because we set the movement precision selection low in order to 
keep the vehicles in a column on the road. We must remember that there are only 16 
possible blue casualties in the scenario, with 6450 observations for each location our 


means will tend to be similar. 


To this point, we have not examined the value that the UAV provides in the 
scenario. Recall from Chapter III that the UAV moves along the route searching for 
enemy agents, if the UAV spots the enemy observer, it will call for long-range fire to 
eliminate the observer. We want to see how the convoy does when the observer is 
eliminated. Of the 25,800 observations, there are 2,203 where the observer is detected 


and eliminated. In Figure 37, we compare the distributions and summary statistics of 
fel 


when the red observer is killed and when not killed. We can plainly see that there is a 
statistical difference in the two means, suggesting the convoy does better when the 
observer is eliminated. Of more interest is the larger proportion of observations where 
casualties were less than or equal to 4. When the observer dies, 50 percent of the 
observations result in 4 or fewer casualties versus the alternate where only 25 percent of 
the observations were 4 or fewer. A final point to make regarding the importance of the 
UAV is the fact that out of the 25,800 total observations there were only 334 where the 
convoy suffered no casualties, of the 334, 97 occurred when the UAV detected the 
observer. We used the following hypothesis test of population proportions to determine 


if there is a statistical difference between the two cases. 


Hi, : p,- P, =9 
i, > P\ — P, #9 
Pre Po 


J pq(/m+1/n) 





Test statistic: z= 


Where p; is the proportion of convoys with no casualties if the observer is killed 
and p2 is the proportion of convoys with no casualties when the observer is not killed. 
From the test we get a z test statistic of 10.445, using an a@ of .005 the corresponding z 
value is 2.58. From this test, we get a p-value, which is the probability that the null 
hypothesis is true, of 1.296 x E~°. This p-value is so minuscule that at any reasonable 
level of @ the null hypothesis should be rejected. The two proportions are clearly not 


equal and we reject the null hypothesis that the two proportions are equal. 


From the test we can conclude that the UAV does provide a significant advantage 
to our convoy and that usage of UAVs for convoy route surveillance should be closely 


examined. 
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Figure 37. Comparison of Data When Red Observer is Eliminated to When Not 
Eliminated, Using Histograms and Summary Statistics. 


From the data set, we notice the relevance and irrelevance of the security vehicles 
inorganic aggressiveness and organic aggressiveness, respectively. Recall that inorganic 
aggressiveness refers to a security vehicles’ desire to pursue the enemy when the 
detection is received from outside its squad. Organic aggressiveness refers to an agents’ 
desire to pursue the enemy when they make the enemy detection themselves. We were 
surprised to see that organic aggressiveness is not a significant factor in the model. After 
careful consideration of the scenario it became apparent what is actually happening and 
seems to be a remnant of the scenario construction. Recall that our security vehicles are 
all separate squads, each with one vehicle. When we conducted our runs, we locked the 
security vehicles together when their relevant factors were varied. This causes them to 


act as a single squad with the same desires. Since an inorganic communications link was 
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established between the three squads, in essence if one detects the enemy, they all detect 
and respond similarly. We reason that due to the model construction the effects of the 
inorganic aggressiveness were dampening those of the agent aggressiveness. In 
hindsight, if we wanted to examine the effects of each security vehicle acting 
independently we should have unlocked the factor settings for organic aggressiveness. 
Though this might seem to be a waste of a factor, it seems to demonstrate the importance 


of the communications links between the blue squads. 


From our exploratory runs and logical thinking, we presumed that vehicle speeds 
would have a significant effect in the scenario. We varied both the security vehicle 
speeds and the logistic vehicle speeds upon enemy contact. One would think that the 
faster the security can close on the enemy and the logistics vehicles can move from the 
kill zone, the fewer casualties we would suffer in the scenario. Neither of the terms 
provide a significant impact on the final model. However, looking closer at the data, we 
can see from the contour plot in Figure 38 that convoy success depends on the speeds of 
the vehicles being relatively close to each other. Meaning that if the security is fast but 
convoy is slow we do bad, and vice versa. The oval in Figure 38 shows a trough on the 
diagonal of the plot that justifies this conclusion. Consequently, in our scenario, the 


security vehicles staying close to the convoy plays a bigger role than speed alone. 
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Figure 38. Contour Plot of Log Speed vs. Security Speed. (Best Viewed in Color) 
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Earlier in this chapter we noted the high number of runs that resulted in 12 blue 
casualties. Looking closer at the instances where this occurred we find it can be 
explained by several occurrences. We plotted the distributions of parameter settings 
against the number of blue casualties when they are 12 and 12 or greater. From the plots, 
we were able to identify the factors that are biased towards a range of settings. Of the 12 
factors, 2 show a significant bias towards a range of settings, see Figure 39. When 
security inorganic aggressiveness is negative and logistic vehicles toward concealment is 
positive, we see the 12 blue casualties occur. In order to figure out why there is a spike at 
12 blue casualties in the data we plotted the same factors against when casualties were 12 
and greater. In Figure 40, nearly the same distributions appear for the two relevant 
factors. At this point, we are convinced that the spike at 12 is an artifact of the scenario 
and not by something occurring out of the ordinary. There are 15 vehicles in the convoy 
that can be killed, three of which are security vehicles that can only be killed by the IED 
or an RPG shooter. When the casualties are 12 or greater, the security tends to avoid 
contact while the logistic vehicles look for concealment. In most cases, it appears that 
either the RPG or the IED at the start of an ambush hits the first security vehicle. At the 
same time, the other two security vehicles fall to the rear of the convoy away from the 
enemy. This causes the RPGs to attack the logistical vehicles seeking concealment. 
Each RPG shooter has only three rounds, and are used on the logistics vehicles in these 
cases. By the time the surviving security vehicles resume movement, there are no other 
weapons capable of killing the security and we see the result of a spike at 12. In most 
cases this resulted in 2 security vehicles and 1 logistics vehicle surviving. Worth 
mentioning, is the fact that fewer red agents are killed in these runs, specifically the RPG, 
who after firing all 3 rounds will retreat. We believe that the spike at 12 is not significant 
and can probably be shifted by a small change in the scenario, for instance, increasing or 
decreasing the number of RPG rounds. Of more importance is what causes the high 


number of blue casualties, which was explained previously in this chapter. 
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Figure 39. Distributions of Security Inorganic Aggressiveness and Logistic Vehicle 
Toward Concealment when there are 12 Blue Casualties. 
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Figure 40. Distributions of Security Inorganic Aggressiveness and Logistic Vehicle 
Toward Concealment when there are 12 or greater Blue Casualties. 
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9. Conclusions 

In this chapter, we have built a model that explains 75 percent of the variance in 
our data set when predicting blue casualties, we have thoroughly examined the significant 
terms in the model and we have highlighted some of the other interesting observations 
found in the data set. Though most of our conclusions are intuitive regarding convoy 
operations, we have shown that it is possible to model convoys using agent-based 
models. We have come to this conclusion by observing extreme effects that were 
purposely planted in the scenario to help validate the outcomes. In the final chapter of 
this thesis, we will summarize our findings and try to relate them to the “real world.” 
Appendix B provides the actual summary statistics from our final regression model. It 
should be noted that interpreting the terms as they relate to our scenario is relevant, using 


an actual regression equation from this scenario to predict in the “real world” is not. 


ie: 
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VI. CONCLUSIONS AND RECOMMENDATIONS 


Over the course of this research, we have seen the end of offensive operations and 
the beginning of the nation building process in Iraq. As the nation building process has 
evolved over the past several months we have witnessed an insurgent force in Iraq 
establish themselves and continually find the means to inflict casualties on the coalition 
forces. The insurgents are evolving and are constantly honing their tactics, techniques 
and procedures. We know that the majority of attacks occurring are directed at coalition 
vehicle convoys. We can assume that future enemies will realize the success of this 


tactic. 


This thesis has taken a simulation and modeling approach to examining the 
tactics, techniques and procedures employed during convoy operations. We have 
examined historical considerations, reviewed the current doctrinal practices, explained 
the chosen modeling environment, data farmed over numerous possible factors and 
applied some common statistical techniques to analyze the results. In this chapter we 
summarize our findings and provide recommendations for each of major phases of this 
thesis. 

A. CONCLUSIONS 

1. Convoy Operations 

We have observed insurgents in Iraq consistently attacking our convoys. We can 
assume that this trend will continue in Iraq and future enemies will adopt the tactic. 
Movement of supplies is an integral part of any military operation and we cannot afford 


to be limited in our movements by small forces with minimal resources. 


In this thesis we have identified some of the important factors surrounding convoy 
operations based on our scenario. We understand that convoy operations in hostile 
situations are subject to an extremely chaotic environment where millions of varying 
factors can produce a wide range of outcomes. It would be impossible to model every 
minute detail of a convoy operation in a single scenario, but this was not our intent. We 
wanted to identify global effects of prominent factors and determine if it was possible to 


model convoys using an ABM. 
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From our scenario we cannot say that traveling at a certain speed is optimal or 
that surveying a convoy route with a UAV will always guarantee positive results. 
However, there are few areas of interest we can conclude from information obtained from 
our scenario: 

e Use of a UAV dramatically reduces the number of casualties suffered. 


e Communications between all parties involved, including the UAV 
significantly increases the probability of success. 


e The convoy composition (i.e. positioning of security vehicles) is 
important. For example, while having combat power massed in most 
combat operations is a benefit, we found in our scenario, when security 
vehicles are massed in one location of the convoy it leads to more 
casualties. 


® The tactics of the logistic vehicles is a bigger determinant of success than 
that of the security vehicles. 


° Speeds of vehicles are relative, it is more important for the convoy to 
operate as unit than as individuals. 


To the operators in the field, we pose these observations as areas to examine in 
the hope that through practical real world experimentation we can find better ways to 
conduct convoy operations. 

ze Agent-Based Models 

Agent-based models provide us with a modeling environment to quickly create 
scenarios based on an abstraction of a complex problem. Coupled with the data farming 
process, the tool is extremely powerful in providing us with insights into a larger and 
more complex problem. Though a powerful tool, caution should be exhibited in their 
use, definitive conclusions and predictions should not be made from the outcome of an 
ABM. Rather, we should seek to increase our understanding of the relevant problem and 
hopefully gain insights from the global behaviors exhibited in the outcomes. Combat 
operations exist in a dynamic and fluid environment where human interactions control 
outcomes. ABMs offer us a tool that attempts to simulate these interactions. 

B. RECOMMENDATIONS 

1. Convoy Operations 

In our research, we have only scratched the surface of the possible factors that can 
be examined to gain insights into convoy operations. If further research is conducted on 


this topic using ABMs, we highly recommend that subject matter experts (SMEs) be 
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intimately involved in creating the scenario. We were able to gain some input but not at 
the level required. Adding SMEs to the process helps to validate the assumptions made 
when modeling and also provides a much needed reality check on the outcomes. It is the 
opinion of the author that SME input and constructive criticisms, coupled with the 
statistical knowledge of an analyst will pay dividends for the Armed Services. 

2. Agent-Based Models 

Though we were successful in modeling a convoy in MANA, there are some 
constraints that could skew some of the output. We found it difficult to model the effects 
of vehicle movements. Unlike the movements of humans on the ground, the movement 
of vehicles is much more constrained by their surroundings. A major factor in convoy 
operations is the dispersion distances between vehicles, we attempted to model this 
factor, but in the end, we do not believe it was effectively modeled providing any useful 
information. Dispersion distances can severely influence the effectiveness of enemy 
weapons and IEDs on convoys, as well as the effectiveness of the convoy weapons on the 
enemy. The low resolution of ABMs inhibits us from examining these types of details, 
but we should remember that ABMs are not designed to look at very detailed physical 
effects. 


It was found that current convoy sizes are relatively small, around 20 vehicles, 
however, during the offensive operations of OIF the logistical convoys tended to be well 
over 50 vehicles. If we wanted to model a larger convoy over bigger terrain area in 
MANA, it would be difficult due to the constraints of the graphical MANA battlefield. 
Perhaps this type of modeling environment is not quite ready for examining a larger 


convoy. 


We pose these observations to the ABM developers not as criticisms, rather as 
insights for them to contemplate in future development of follow-on versions of the 
models. 

C. AREAS FOR FOLLOW ON RESEARCH 

There are thousands of problems within the military that can be examined using 
ABMs. During the course of this research, we found numerous topics within the area of 
convoy operations that can and need to be examined. The following is a list of topics that 


could be examined either independent of this thesis or as a follow-on to this thesis. 
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° Alter the size of convoys with varying numbers of security vehicles and 
weapon mixes. 


° Examine convoy operations in a more urban terrain. 


e Thoroughly model and examine the effects of communication amongst the 
convoy vehicles. We assumed perfect communications in our scenario, 
this is far from reality. 


° Model convoy operations using emerging technologies, such as gunfire 
detection and location (GDL) devices and frequency jamming systems to 
counter IEDs. 


We leave the reader with one final thought; during the Cold War, control of the 
skies was of the utmost importance during a conflict. In the current global environment, 


control of the roads is now at the top of our list. 


82 


APPENDIX A. FACTORS EXPLORED 


Listed are the 11 factors that where explored in this thesis. We identify the factor, 


state in which it is varied, the range of the factor, the MANA definition from version 


3.0.35 user’s manual [Anderson, et al., 2004], and finally our interpretation of how the 


factor relates to the “real world.” 


A. SECURITY LOCATION 


State — Security vehicles are placed in their respective positions (listed 
below) at the start of each run. We use four separate scenarios to examine 
locations. 


e Location 1 — Per doctrine [MCRP 4-11.3F, 2001], security is 
evenly distributed within the convoy, | front, 1 middle and 1 in the 
rear of the convoy. 


e Location 2 — 2 in front and 1 in the rear of convoy. 
e Location 3 — 1 in front and 2 in the rear of convoy 
e Location 4 — All 3 security vehicles in middle of convoy. This 


location was used to identify the validity of the scenario; we expect 
this location to result in more blue casualties. 


Range — Not applicable for this factor. 
MANA Definition — Not applicable for this factor. 


Real World Interpretation and Justification — In our research we had seen 
reports from the AOR instructing the use of security vehicle locations that 
were not in agreement with doctrinal practices. We wanted to identify if 
altering security vehicle locations would impact mission success in our 
scenario. 


B. SECURITY SPEED 


State — Spare 1, which is the enemy contact state. 
Range — (90 to 250) 


MANA Definition — The number of cells an agent can move in a single 
time step divided by 100. Speed can be set between (0-1000) and can 
change in altering states. 


Real World Interpretation and Justification — Security vehicle movement 
speed. We wanted to see if the security vehicles were able to close on the 
enemy faster, if it would make a difference in results. A speed of 100 was 
set as the baseline for vehicles. We assumed the average convoy speed 
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would be around 30 mph. A speed of 250 would represent 80 mph, which 
is extreme, but we wanted to see if extreme cases make a difference. A 
convoy speed of 100 is roughly 3 times the speed of the enemy ambushers. 


SECURITY VEHICLE TIME IN STATE 


State — Spare 1. 
Range — (1 to 132) 


MANA Definition — Number of time steps a squad will stay in a state once 
it has been triggered. Ifa state of 0 is set the state will never be entered. 
The time can be set between 0 and 99,999,999, 


Real World Interpretation and Justification — The duration of time security 
vehicles will pursue an enemy or act on the corresponding tactics of the 
relevant state. Time in state was used to identify if a prolonged pursuit of 
the enemy contributed to mission success or failure. 


SECURITY VEHICLE AGENT SA AGGRESSIVENESS (ENEMY) 


State — Spare 1. 
Range — (-33 to 33) 


MANA Definition — Agent SA refers to an agents’ propensity towards 
personalities based on information it has obtained in the current time step. 


Real World Interpretation and Justification — Corresponds to how much 
the security wants to engage an enemy. We used a negative value to 
identify extremes, where the security actually stays away from enemies. 


SECURITY INORGANIC SA AGGRESSIVENESS (ENEMY) 


State — Spare 1. 
Range — (-33 to 33) 


MANA Definition — Inorganic SA refers to a squad’s response based on 
contact information received from agents outside of the relevant squad. 


Real World Interpretation and Justification — Corresponds to how much 
the security vehicles want to engage an enemy based on information 
received from other squads or units, could be from other security vehicles, 
logistical vehicles or a UAV. Our reasoning was that this factor would 
show the value of communications and situational awareness. 


SECURITY VEHICLES STAY WITH AGENT SA FRIENDS 


State — Spare 1. 
Range — (-33 to 33) 


MANA Definition — Refers to an agent’s propensity towards other agents 
of the same allegiance within sensor range. 
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Real World Interpretation and Justification — Represents how much the 
security vehicles want to stay with and protect other vehicles in their 
proximity. The negative value was used to show extreme cases where the 
security leaves the logistical vehicles. However, if this factor is coupled 
with a negative propensity towards the enemy we see an extreme case that 
would never happen in the “real world.” 


SECURITY VEHICLE CLASSIFICATION/DETECTION RANGE 


State — Default 
Range — (10 to 100) 


MANA Definition — Refers to the radius in cells that an agent can see 
other entities around them. Classification refers to the radius in cells that 
an agent can classify detections as friendly, enemy or neutral. In our 
scenario the two are locked together. 


Real World Interpretation and Justification — Represents the visibility 
limitations that vehicles in a column coupled with terrain, weather and 
movement might cause the convoy. The default state for security vehicles 
was used to see if earlier enemy detections by the security vehicles cause a 
significant difference. 


LOGISTIC VEHICLES TOWARD CONCEALMENT 


State — Spare 1. 
Range — (-33 to 33) 


MANA Definition — Refers to an agents desire to seek terrain areas that 
provide concealment. 


Real World Interpretation and Justification — Can be interpreted as the 
logistical vehicles within the convoy wanting to seek out a concealed area 
rather than drive through an ambush upon contact. A negative value will 
cause the logistical vehicles to remain on the route and drive through an 
ambush while a positive value will cause them to seek concealment upon 
enemy contact. 


LOGISTIC VEHICLE SPEED 


State — Spare 1. 
Range — (90 to 250) 


MANA Definition - The number of cells an agent can move in a single 
time step divided by 100. Speed can be set between (0 and 1000) and can 
change in altering states. 


Real World Interpretation and Justification — Upon enemy contact this can 
relate to the logistical vehicles speeding up through an ambush. Selection 
of speeds is same as reasoning for security vehicle speeds. 
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LOGISTIC VEHICLES TIME IN STATE 


State — Spare 1. 
Range — (1 to 132) 


MANA Definition - Number of time steps a squad will stay in a state once 
it has been triggered. Ifa state of 0 is set the state will never be entered. 
The time can be set between 0 and 99,999,999. 


Real World Interpretation and Justification — Similar to the security 
vehicles time in state factor. If a convoy is ambushed and seeks out 
concealment, this factor determines how long they will wait before 
proceeding along the route. We wanted to identify if it was better for the 
logistical vehicles to give the security time to pursue the enemy or better 
to quickly resume travel along the route. 


LOGISTIC VEHICLES DETECTION/CLASSIFICATION RANGE 


State — Spare 1. 
Range — (10 to 100) 


MANA Definition - Refers to the radius in cells that an agent can see other 
entities around them. Classification refers to the radius in cells that an 
agent can classify detections as friendly, enemy or neutral. In our scenario 
the two are locked together. 


Real World Interpretation and Justification — Similar to the security 
vehicles detection and classification range with one exception. This factor 
was varied in the enemy contact state to determine if good situational 
awareness when ambushed significantly impacted the results. It should be 
noted that when this range is high, the security vehicles are also benefiting 
from the detections made through the inorganic communication links. 


UAV DETECTION AND CLASSIFICATION RANGE 


State — Default. The UAV does not remain in an enemy contact state for 
any prolonged period. 


Range — (10 to 120) 


MANA Definition - Refers to the radius in cells that an agent can see other 
entities around them. Classification refers to the radius in cells that an 
agent can classify detections as friendly, enemy or neutral. In our 
scenario, the two are locked together. 


Real World Interpretation and Justification — Simply represents how much 
a UAV can see below them on the ground. A value of 10 is extremely 
limited, but a value of 120 as it relates to the “real world’ would be an 
extreme case. The intent was to identify whether or not an increased 
awareness of enemy locations assisted the convoy. 
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APPENDIX B. FINAL MULTIPLE LINEAR REGRESSION 
MODEL SUMMARY STATISTICS 


Summary of Fit 


RSquare 

RSquare Adj 

Root Mean Square Error 
Mean of Response 
Observations (or Sum Wets) 


Analysis of Variance 


Source 
Model 
Error 

C. Total 


Parameter Estimates 


Term 


Intercept 

Sec Loc4 

Sec Time in State 

Sec agent aggressive 

Sec Inorg SA aggressiveness 

Sec stay with friends 

Log twd conceal 

Log class/detect range 

UAV class/detect range 

(Sec Time in State-66.5039)*(Sec stay with friends-0.03101) 

(Sec Time in State-66.5039)*(Log twd conceal-0.03101) 

(Sec Inorg SA aggressiveness-0.03101)*(Log twd conceal-0.03101) 
(Log class/detect range-55.031)*(UAV class/detect range-65.03 1) 
(Sec stay with friends-0.03101)*(Sec stay with friends-0.03101) 
(Log twd conceal-0.03101)*(Log twd conceal-0.03101) 
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0.7421 
0.7349 
1.1918 
6.2403 
516 


DF 

14 
501 
515 


Estimate 


6.7827 
0.5343 
0.0100 
0.0137 
-0.0467 
-0.0144 
0.0482 
-0.0183 
-0.0086 
-0.0004 
0.0005 
-0.0029 
0.0004 
-0.0010 
0.0015 


Sum of Squares 
2048.3292 
711.6735 
2760.0027 


Std Error 


0.2022 
0.1212 
0.0013 
0.0026 
0.0026 
0.0026 
0.0026 
0.0019 
0.0016 
0.0001 
0.0001 
0.0002 
0.0001 
0.0002 
0.0002 


Mean Square 
1.46E+02 
1.42E+00 





t Ratio 


3.35E+01 
4.41E+00 
7A9E+00 
5.20E+00 
-1.77E+01 
-5.48E+00 
1.83E+01 
-9.42E+00 
-5.41E+00 
-5.24E+00 
6.69E+00 
-1.94E+01 
5.95E+00 
-6.21E+00 
9.20E+00 











F Ratio 
1.03E+02 
2.22E-137 


Prob>|t| 


3.44E-130 
1.27E-05 
3.04E-13 
2.91E-07 
5.16E-55 
6.73E-08 
9.96E-58 
1.61E-19 
9.63E-08 
2.41E-07 
5.83E-11 
9.31E-63 
5.15E-09 
1.14E-09 
1.00E-18 
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